[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:07 dist-test-slave-nfb5 krb5kdc[25311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:07 dist-test-slave-nfb5 krb5kdc[25311](info): set up 2 sockets
Aug 08 07:00:07 dist-test-slave-nfb5 krb5kdc[25311](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 07:00:08.956339 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:09.080034 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37431
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:37431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:09.164225 25337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:09.164463 25337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:09.164499 25337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:09.165633 25337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:09.165706 25337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:09.165735 25337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:09.165766 25337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:09.165795 25337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:09.167428 25337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:37431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:09.167713 25337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:09.167951 25337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:09.174093 25342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.174078 25343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:09.174163 25337 server_base.cc:1034] running on GCE node
W20260808 07:00:09.174078 25345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:09.174657 25337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:09.175102 25337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:09.176259 25337 hybrid_clock.cc:648] HybridClock initialized: now 1786172409176245 us; error 84 us; skew 500 ppm
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:09.178735 25337 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:09.179243 25337 webserver.cc:460] Webserver started at http://127.24.181.126:41147/ using document root <none> and password file <none>
I20260808 07:00:09.179472 25337 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:09.179540 25337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:09.179698 25337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:09.180862 25337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "6c16a2953f4141239fc5e96237c9e109"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "1cb7e83c4e22ff420ed29a40302b4669"
server_key_iv: "63b25cc92cc8490d811158c9e362981a"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.181243 25337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "6c16a2953f4141239fc5e96237c9e109"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "1cb7e83c4e22ff420ed29a40302b4669"
server_key_iv: "63b25cc92cc8490d811158c9e362981a"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.182595 25337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:09.183511 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.183681 25337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.183786 25337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "6c16a2953f4141239fc5e96237c9e109"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "1cb7e83c4e22ff420ed29a40302b4669"
server_key_iv: "63b25cc92cc8490d811158c9e362981a"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.183868 25337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:09.202811 25337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:09.207036 25337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:09.207203 25337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:09.212978 25337 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:37431
I20260808 07:00:09.213091 25427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:37431 every 8 connection(s)
I20260808 07:00:09.213488 25337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:09.215530 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25337
I20260808 07:00:09.215633 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:09.216333 25301 external_mini_cluster.cc:1428] Setting key 369dc2166408d56824f8b06a1a016c43
I20260808 07:00:09.217424 25428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:09.220079 25428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109: Bootstrap starting.
I20260808 07:00:09.220741 25428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:09.221047 25428 log.cc:824] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:09.221773 25428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109: No bootstrap required, opened a new log
I20260808 07:00:09.224031 25428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } }
I20260808 07:00:09.224251 25428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:09.224308 25428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16a2953f4141239fc5e96237c9e109, State: Initialized, Role: FOLLOWER
I20260808 07:00:09.224471 25428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } }
I20260808 07:00:09.224574 25428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:09.224614 25428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:09.224673 25428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:09.225535 25428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } }
I20260808 07:00:09.225675 25428 leader_election.cc:302] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c16a2953f4141239fc5e96237c9e109; no voters:
I20260808 07:00:09.225883 25428 leader_election.cc:288] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:09.225908 25435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:09.226040 25435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [term 1 LEADER]: Becoming Leader. State: Replica: 6c16a2953f4141239fc5e96237c9e109, State: Running, Role: LEADER
I20260808 07:00:09.226176 25428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:09.226240 25435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } }
I20260808 07:00:09.226872 25436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c16a2953f4141239fc5e96237c9e109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } } }
I20260808 07:00:09.227005 25436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:09.227203 25438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c16a2953f4141239fc5e96237c9e109. Latest consensus state: current_term: 1 leader_uuid: "6c16a2953f4141239fc5e96237c9e109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c16a2953f4141239fc5e96237c9e109" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37431 } } }
I20260808 07:00:09.227252 25443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:09.227283 25438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:09.227802 25443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:09.229663 25443 catalog_manager.cc:1347] Generated new cluster ID: 9524756b125e4453897328ac350b3822
I20260808 07:00:09.230331 25443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:09.236232 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.218892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.24.181.126:37431)
0808 07:00:09.219105 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.219116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.219153 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:00:09.221183 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.221190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.221244 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.221399 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.223079 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.223975 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.225257 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.225713 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.225837 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.231735 (+ 5898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.231739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.231739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.231750 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.233882 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.234398 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.234401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.234401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.234511 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.234797 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.234798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.234798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.235137 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.235315 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.235640 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.235784 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260808 07:00:09.255158 25443 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:09.255915 25443 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:09.263739 25443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c16a2953f4141239fc5e96237c9e109: Generated new TSK 0
I20260808 07:00:09.263953 25443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:09.295840 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:09.396399 25466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:09.396608 25466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:09.396634 25466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:09.397728 25466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:09.397764 25466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:09.397791 25466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:09.399282 25466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:09.399464 25466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:09.399636 25466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:09.405078 25476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.405086 25477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.405114 25481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:09.405444 25466 server_base.cc:1034] running on GCE node
I20260808 07:00:09.405611 25466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:09.405810 25466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:09.406989 25466 hybrid_clock.cc:648] HybridClock initialized: now 1786172409406980 us; error 40 us; skew 500 ppm
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:09.409785 25466 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:09.410158 25466 webserver.cc:460] Webserver started at http://127.24.181.65:39769/ using document root <none> and password file <none>
I20260808 07:00:09.410389 25466 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:09.410436 25466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:09.410578 25466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:09.411723 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "7e1b20df77f348b5ad79c5265e2b0324"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "838ef359a9ed18be53282bf3a7af070e"
server_key_iv: "03ac2a96b572b622c7d1432f5b0e328b"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.412112 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "7e1b20df77f348b5ad79c5265e2b0324"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "838ef359a9ed18be53282bf3a7af070e"
server_key_iv: "03ac2a96b572b622c7d1432f5b0e328b"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.413757 25466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:00:09.414498 25492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.414793 25466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:09.414913 25466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "7e1b20df77f348b5ad79c5265e2b0324"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "838ef359a9ed18be53282bf3a7af070e"
server_key_iv: "03ac2a96b572b622c7d1432f5b0e328b"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.415014 25466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:09.440948 25466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:09.443326 25466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:09.443521 25466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:09.443881 25466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:09.444262 25466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:09.444319 25466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.444356 25466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:09.444404 25466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.454979 25466 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:38255
I20260808 07:00:09.455156 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:38255 every 8 connection(s)
I20260808 07:00:09.455435 25466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:09.462059 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25466
I20260808 07:00:09.462139 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:09.462348 25301 external_mini_cluster.cc:1428] Setting key a9a4d97383c73294790201d98d852d24
I20260808 07:00:09.465449 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.456315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:52097 (local address 127.24.181.126:37431)
0808 07:00:09.456376 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.456377 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.456622 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:00:09.457104 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.457105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.457116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.457175 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.458253 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.458755 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.459819 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.460026 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.460095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.462233 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.462235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.462235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.462243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.464010 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.464435 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.464437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.464437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.464499 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.464768 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.464769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.464770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.464895 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.464966 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.465268 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.465358 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:00:09.465698 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.456281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37431 (local address 127.24.181.65:52097)
0808 07:00:09.456493 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:09.456530 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:09.456993 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:09.457235 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:09.457238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:09.457964 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:09.458153 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.458158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.458865 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.458866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.459726 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.459728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.460113 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.460114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.460212 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.460670 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:09.460677 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:09.462077 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:09.464115 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.464123 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.464125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.464356 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.464574 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.464575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.464575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.464692 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.464969 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:09.464976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:09.465131 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 07:00:09.465244 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:00:09.465409 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":149,"threads_started":1}
I20260808 07:00:09.466252 25650 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37431
I20260808 07:00:09.466373 25650 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:09.466568 25650 heartbeater.cc:511] Master 127.24.181.126:37431 requested a full tablet report, sending...
I20260808 07:00:09.467144 25380 ts_manager.cc:194] Registered new tserver with Master: 7e1b20df77f348b5ad79c5265e2b0324 (127.24.181.65:38255)
I20260808 07:00:09.467753 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:52097
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:09.490438 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:09.604367 25656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:09.604571 25656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:09.604596 25656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:09.605860 25656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:09.605902 25656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:09.605930 25656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:09.607849 25656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:09.608112 25656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:09.608369 25656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:09.614218 25664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.614264 25663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.614290 25666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:09.614837 25656 server_base.cc:1034] running on GCE node
I20260808 07:00:09.615070 25656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:09.615340 25656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:09.616497 25656 hybrid_clock.cc:648] HybridClock initialized: now 1786172409616487 us; error 28 us; skew 500 ppm
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:09.620290 25656 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:09.620853 25656 webserver.cc:460] Webserver started at http://127.24.181.66:46089/ using document root <none> and password file <none>
I20260808 07:00:09.621121 25656 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:09.621184 25656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:09.621322 25656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:09.622782 25656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "e910fa5837cb451a8d38b08add50989a"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "da62f6dd176dfaf9bc6972251602c1ed"
server_key_iv: "3e53a4a8b3d5486cc0f2f0910acfbed8"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.623243 25656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "e910fa5837cb451a8d38b08add50989a"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "da62f6dd176dfaf9bc6972251602c1ed"
server_key_iv: "3e53a4a8b3d5486cc0f2f0910acfbed8"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.625046 25656 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:00:09.626070 25678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.626277 25656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:09.626366 25656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "e910fa5837cb451a8d38b08add50989a"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "da62f6dd176dfaf9bc6972251602c1ed"
server_key_iv: "3e53a4a8b3d5486cc0f2f0910acfbed8"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.626434 25656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:09.683493 25656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:09.685622 25656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:09.685853 25656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:09.686124 25656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:09.686488 25656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:09.686548 25656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.686612 25656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:09.686662 25656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.697866 25656 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:35235
I20260808 07:00:09.697917 25833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:35235 every 8 connection(s)
I20260808 07:00:09.698336 25656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:09.708240 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25656
I20260808 07:00:09.708364 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:09.708546 25301 external_mini_cluster.cc:1428] Setting key f048dcf73d47d0d39643580f3c28ebc7
I20260808 07:00:09.708762 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.699247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:46725 (local address 127.24.181.126:37431)
0808 07:00:09.699299 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.699302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.699654 (+ 352us) server_negotiation.cc:407] Connection header received
0808 07:00:09.700391 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.700392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.700402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.700457 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.701621 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.702104 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.703271 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.703458 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.703514 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.705780 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.705782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.705782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.705796 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.707261 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.707672 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.707674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.707674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.707731 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.707988 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.707989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.707990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.708156 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.708233 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.708556 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.708659 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:09.709038 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.699209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37431 (local address 127.24.181.66:46725)
0808 07:00:09.699475 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:09.699511 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:09.700297 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:09.700523 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:09.700526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:09.701240 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:09.701538 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.701544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.702189 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.702190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.703192 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.703195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.703531 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.703532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.703687 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.704219 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:09.704227 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:09.705666 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:09.707349 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.707355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.707358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.707584 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.707797 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.707797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.707798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.707911 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.708235 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:09.708242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:09.708413 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 07:00:09.708527 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:00:09.708756 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":122,"threads_started":1}
I20260808 07:00:09.709611 25835 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37431
I20260808 07:00:09.709717 25835 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:09.709920 25835 heartbeater.cc:511] Master 127.24.181.126:37431 requested a full tablet report, sending...
I20260808 07:00:09.710371 25380 ts_manager.cc:194] Registered new tserver with Master: e910fa5837cb451a8d38b08add50989a (127.24.181.66:35235)
I20260808 07:00:09.710947 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:46725
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:09.737526 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:09.867836 25840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:09.868072 25840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:09.868115 25840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:09.869323 25840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:09.869390 25840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:09.869436 25840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:09.871132 25840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:09.871405 25840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:09.871627 25840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:09.878075 25846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.877969 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:09.878006 25852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:09.878302 25840 server_base.cc:1034] running on GCE node
I20260808 07:00:09.878541 25840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:09.878786 25840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:09.879951 25840 hybrid_clock.cc:648] HybridClock initialized: now 1786172409879956 us; error 46 us; skew 500 ppm
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:09.882480 25840 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:09.882908 25840 webserver.cc:460] Webserver started at http://127.24.181.67:35735/ using document root <none> and password file <none>
I20260808 07:00:09.883206 25840 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:09.883275 25840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:09.883438 25840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:09.884490 25840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "2c063c4312b14aca92671fdacdf67d82"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "8cba9f1f94bd66d07677c6cfa538641e"
server_key_iv: "1e700246447d61016e3981fc5aace4d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.884860 25840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "2c063c4312b14aca92671fdacdf67d82"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "8cba9f1f94bd66d07677c6cfa538641e"
server_key_iv: "1e700246447d61016e3981fc5aace4d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.886512 25840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:09.887399 25862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.887581 25840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:09.887683 25840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "2c063c4312b14aca92671fdacdf67d82"
format_stamp: "Formatted at 2026-08-08 07:00:09 on dist-test-slave-nfb5"
server_key: "8cba9f1f94bd66d07677c6cfa538641e"
server_key_iv: "1e700246447d61016e3981fc5aace4d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:09.887761 25840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:09.901932 25840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:09.904259 25840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:09.904438 25840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:09.904670 25840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:09.904979 25840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:09.905042 25840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.905081 25840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:09.905108 25840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:09.917778 25840 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:44627
I20260808 07:00:09.917924 26015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:44627 every 8 connection(s)
I20260808 07:00:09.918231 25840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:09.924443 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25840
I20260808 07:00:09.924549 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:09.924789 25301 external_mini_cluster.cc:1428] Setting key a690b535be974cfa5c5dece58f124e34
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:09.932381 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.919138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:59161 (local address 127.24.181.126:37431)
0808 07:00:09.919201 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.919203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.919541 (+ 338us) server_negotiation.cc:407] Connection header received
0808 07:00:09.920271 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.920273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.920284 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.920343 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.921531 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.922281 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.924067 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.924391 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.924476 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.928043 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.928045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.928046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.928054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.930292 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.930898 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.930899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.930900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.931002 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.931396 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.931397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.931406 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.931609 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.931717 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.932136 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.932257 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:00:09.932672 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.919088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37431 (local address 127.24.181.67:59161)
0808 07:00:09.919389 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:09.919426 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:09.920151 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:09.920396 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:09.920399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:09.921116 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:09.921410 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.921415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.922389 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.922390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.923969 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:09.923972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.924552 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.924553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:09.924832 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.925642 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:09.925652 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:09.927942 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:09.930395 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.930402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.930403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.930787 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.931127 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:09.931128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:09.931129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:09.931291 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:09.931739 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:09.931746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:09.931947 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 07:00:09.932083 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:00:09.932300 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":166,"threads_started":1}
I20260808 07:00:09.933260 26018 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37431
I20260808 07:00:09.933372 26018 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:09.933597 26018 heartbeater.cc:511] Master 127.24.181.126:37431 requested a full tablet report, sending...
I20260808 07:00:09.934008 25380 ts_manager.cc:194] Registered new tserver with Master: 2c063c4312b14aca92671fdacdf67d82 (127.24.181.67:44627)
I20260808 07:00:09.934475 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:59161
I20260808 07:00:09.937778 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:09.964071 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.955337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.24.181.65:38255)
0808 07:00:09.955496 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.955501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.955554 (+ 53us) server_negotiation.cc:407] Connection header received
0808 07:00:09.955600 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.955601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.955615 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.955688 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.956386 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.957088 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.957849 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.958156 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.958211 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.960869 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.960878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.960878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.960884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.962546 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.963054 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.963057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.963057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.963114 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.963372 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.963372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.963372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.963508 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.963616 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.963826 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.963906 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
W20260808 07:00:09.964498 25528 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46266
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:00:09 dist-test-slave-nfb5 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:09.988399 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:09.981187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.24.181.65:38255)
0808 07:00:09.981248 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:09.981250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:09.981275 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:00:09.981363 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:09.981364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:09.981375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:09.981424 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:09.981991 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.982466 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.983251 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:09.983493 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:09.983549 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:09.985724 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:09.985726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:09.985726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:09.985732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:09.987308 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.987729 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.987730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.987730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.987780 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:09.987946 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:09.987946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:09.987947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:09.988053 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:09.988127 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:09.988221 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:00:09.988282 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:00:09.988981 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25466
I20260808 07:00:09.998903 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25656
I20260808 07:00:10.006367 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25840
I20260808 07:00:10.016001 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25337
2026-08-08T07:00:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2426 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:10 dist-test-slave-nfb5 krb5kdc[26037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:10 dist-test-slave-nfb5 krb5kdc[26037](info): set up 2 sockets
Aug 08 07:00:10 dist-test-slave-nfb5 krb5kdc[26037](info): commencing operation
krb5kdc: starting...
W20260808 07:00:11.258316 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:11.377914 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44887
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:41881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:11.460321 26055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:11.460521 26055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:11.460546 26055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:11.461733 26055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:11.461777 26055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:11.461791 26055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:11.461803 26055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:11.461817 26055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:11.463397 26055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:41881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:11.463662 26055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:11.463888 26055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:11.468842 26068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:11.468856 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:11.469012 26055 server_base.cc:1034] running on GCE node
W20260808 07:00:11.468856 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:11.469434 26055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:11.469684 26055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:11.470845 26055 hybrid_clock.cc:648] HybridClock initialized: now 1786172411470834 us; error 29 us; skew 500 ppm
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:11.473789 26055 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:11.474223 26055 webserver.cc:460] Webserver started at http://127.24.181.126:34811/ using document root <none> and password file <none>
I20260808 07:00:11.474500 26055 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:11.474548 26055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:11.474678 26055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:11.475790 26055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "94d01aae34474200abaf2c30a8b2fef5"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "1afcb33307ecfc9b4fce3db5a8a79ac4"
server_key_iv: "115e527022e74998e32fd47010e048a6"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.476146 26055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "94d01aae34474200abaf2c30a8b2fef5"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "1afcb33307ecfc9b4fce3db5a8a79ac4"
server_key_iv: "115e527022e74998e32fd47010e048a6"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.477543 26055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:00:11.478375 26076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.478608 26055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:11.478724 26055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "94d01aae34474200abaf2c30a8b2fef5"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "1afcb33307ecfc9b4fce3db5a8a79ac4"
server_key_iv: "115e527022e74998e32fd47010e048a6"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.478798 26055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:11.494194 26055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:11.496639 26055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:11.496798 26055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:11.500901 26055 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44887
I20260808 07:00:11.500981 26155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44887 every 8 connection(s)
I20260808 07:00:11.501319 26055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:11.502257 26156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:11.502846 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 26055
I20260808 07:00:11.502956 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:11.503132 25301 external_mini_cluster.cc:1428] Setting key 30d699192dc6d6b165e4179f828db0ee
I20260808 07:00:11.505409 26156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5: Bootstrap starting.
I20260808 07:00:11.506012 26156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:11.506366 26156 log.cc:824] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:11.507175 26156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5: No bootstrap required, opened a new log
I20260808 07:00:11.509778 26156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } }
I20260808 07:00:11.509932 26156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:11.509982 26156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d01aae34474200abaf2c30a8b2fef5, State: Initialized, Role: FOLLOWER
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:11.510115 26156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } }
I20260808 07:00:11.510224 26156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:11.510265 26156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:11.510313 26156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:11.511016 26156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } }
I20260808 07:00:11.511135 26156 leader_election.cc:302] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d01aae34474200abaf2c30a8b2fef5; no voters:
I20260808 07:00:11.511320 26156 leader_election.cc:288] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:11.511384 26163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:11.511526 26163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [term 1 LEADER]: Becoming Leader. State: Replica: 94d01aae34474200abaf2c30a8b2fef5, State: Running, Role: LEADER
I20260808 07:00:11.511585 26156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:11.511873 26163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } }
I20260808 07:00:11.512241 26164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d01aae34474200abaf2c30a8b2fef5. Latest consensus state: current_term: 1 leader_uuid: "94d01aae34474200abaf2c30a8b2fef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } } }
I20260808 07:00:11.512329 26164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:11.512502 26165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d01aae34474200abaf2c30a8b2fef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d01aae34474200abaf2c30a8b2fef5" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44887 } } }
I20260808 07:00:11.512586 26165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:11.513047 26174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:11.513533 26174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:11.514963 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:11.503678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.24.181.126:44887)
0808 07:00:11.503851 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:11.503859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:11.503884 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:00:11.504297 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:11.504303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:11.504346 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:11.504459 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:11.505257 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.506175 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.507183 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.507635 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.507770 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:11.510677 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:11.510681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:11.510681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:11.510691 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:11.512826 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.513277 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.513280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.513280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.513397 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.513646 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.513647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.513647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.513959 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:11.514105 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:11.514582 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 07:00:11.514723 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260808 07:00:11.515492 26174 catalog_manager.cc:1347] Generated new cluster ID: bff7e6ebabc04f70a354ca22e5eb3e7c
I20260808 07:00:11.515558 26174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:11.523890 26174 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:11.524564 26174 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:11.528142 26174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d01aae34474200abaf2c30a8b2fef5: Generated new TSK 0
I20260808 07:00:11.528303 26174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:11.560055 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:41881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:11.661221 26196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:11.661434 26196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:11.661460 26196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:11.662600 26196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:11.662636 26196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:11.662669 26196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:11.664316 26196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:41881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:11.664510 26196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:11.664695 26196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:11.671028 26204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:11.671092 26205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:11.671255 26208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:11.671454 26196 server_base.cc:1034] running on GCE node
I20260808 07:00:11.671731 26196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:11.671976 26196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:11.673146 26196 hybrid_clock.cc:648] HybridClock initialized: now 1786172411673140 us; error 37 us; skew 500 ppm
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:11.676422 26196 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:11.676903 26196 webserver.cc:460] Webserver started at http://127.24.181.65:44267/ using document root <none> and password file <none>
I20260808 07:00:11.677127 26196 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:11.677196 26196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:11.677354 26196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:11.678665 26196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8c251ec276674cf2a999671fe8c766e1"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "8a6799bd72c7467301ef54a574b0d843"
server_key_iv: "d60a0f8d167f948d3220b733b7af30ca"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.679149 26196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8c251ec276674cf2a999671fe8c766e1"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "8a6799bd72c7467301ef54a574b0d843"
server_key_iv: "d60a0f8d167f948d3220b733b7af30ca"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.680864 26196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:11.681849 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.682047 26196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:11.682161 26196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8c251ec276674cf2a999671fe8c766e1"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "8a6799bd72c7467301ef54a574b0d843"
server_key_iv: "d60a0f8d167f948d3220b733b7af30ca"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.682253 26196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:11.718333 26196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:11.720817 26196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:11.721045 26196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:11.721426 26196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:11.721828 26196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:11.721889 26196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.721925 26196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:11.721975 26196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.736855 26196 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:33413
I20260808 07:00:11.736986 26381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:33413 every 8 connection(s)
I20260808 07:00:11.737285 26196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:11.746080 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 26196
I20260808 07:00:11.746176 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:11.746357 25301 external_mini_cluster.cc:1428] Setting key a04db39758ed6c592bc57e8f5e9af269
I20260808 07:00:11.749176 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:11.738192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:49893 (local address 127.24.181.126:44887)
0808 07:00:11.738264 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:11.738266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:11.738632 (+ 366us) server_negotiation.cc:407] Connection header received
0808 07:00:11.739828 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:11.739829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:11.739841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:11.739895 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:11.740653 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.741151 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.742983 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.743197 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.743254 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:11.745832 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:11.745833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:11.745833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:11.745840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:11.747773 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.748164 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.748165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.748166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.748233 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.748473 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.748474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.748475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.748622 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:11.748691 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:11.748991 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:00:11.749087 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:00:11.749370 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:11.738149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44887 (local address 127.24.181.65:49893)
0808 07:00:11.738487 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:11.738539 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:11.739732 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:11.739950 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:11.739953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:11.740258 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:11.740562 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:11.740567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.741229 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.741230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:11.742883 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:11.742886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.743273 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.743274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:11.743408 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:11.744013 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:11.744024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:11.745740 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:11.747858 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:11.747864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:11.747866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:11.748088 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:11.748299 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:11.748300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:11.748300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:11.748410 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:11.748685 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:11.748689 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:11.748843 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:00:11.748950 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:00:11.749109 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":179,"threads_started":1}
I20260808 07:00:11.750106 26382 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44887
I20260808 07:00:11.750221 26382 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:11.750412 26382 heartbeater.cc:511] Master 127.24.181.126:44887 requested a full tablet report, sending...
I20260808 07:00:11.751030 26100 ts_manager.cc:194] Registered new tserver with Master: 8c251ec276674cf2a999671fe8c766e1 (127.24.181.65:33413)
I20260808 07:00:11.751662 26100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:49893
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:11.775409 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:41881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:11.899468 26387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:11.899674 26387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:11.899703 26387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:11.900944 26387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:11.900983 26387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:11.901010 26387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:11.903190 26387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:41881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:11.903414 26387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:11.903617 26387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:11.910776 26402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:11.910964 26395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:11.911011 26398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:11.911263 26387 server_base.cc:1034] running on GCE node
I20260808 07:00:11.911525 26387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:11.911839 26387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:11.913013 26387 hybrid_clock.cc:648] HybridClock initialized: now 1786172411912982 us; error 44 us; skew 500 ppm
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:11.916481 26387 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:11.917032 26387 webserver.cc:460] Webserver started at http://127.24.181.66:38861/ using document root <none> and password file <none>
I20260808 07:00:11.917291 26387 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:11.917364 26387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:11.917515 26387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:11.918774 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "4222b0a05b364500b191044ea5cab067"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "93d3d2d2d83aa28c395d58f334706c35"
server_key_iv: "4d5deed04f6923726a872aa913c3d496"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.919210 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "4222b0a05b364500b191044ea5cab067"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "93d3d2d2d83aa28c395d58f334706c35"
server_key_iv: "4d5deed04f6923726a872aa913c3d496"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.920750 26387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:11.921579 26410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.921797 26387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:11.921918 26387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "4222b0a05b364500b191044ea5cab067"
format_stamp: "Formatted at 2026-08-08 07:00:11 on dist-test-slave-nfb5"
server_key: "93d3d2d2d83aa28c395d58f334706c35"
server_key_iv: "4d5deed04f6923726a872aa913c3d496"
server_key_version: "encryptionkey@0"
I20260808 07:00:11.922005 26387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:11.950634 26387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:11.953171 26387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:11.953400 26387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:11.953770 26387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:11.954165 26387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:11.954233 26387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.954278 26387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:11.954303 26387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:11.968451 26387 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:40045
I20260808 07:00:11.968555 26567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:40045 every 8 connection(s)
I20260808 07:00:11.968931 26387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:11.972630 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 26387
I20260808 07:00:11.972729 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:11.972895 25301 external_mini_cluster.cc:1428] Setting key b9f9f8f8f21088a6137772d91e5a461f
Aug 08 07:00:11 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:11.981957 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:11.969930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37881 (local address 127.24.181.126:44887)
0808 07:00:11.970035 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:11.970038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:11.970392 (+ 354us) server_negotiation.cc:407] Connection header received
0808 07:00:11.971773 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:11.971775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:11.971785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:11.971833 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:11.972627 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.973345 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.974620 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.974970 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.975074 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:11.978132 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:11.978133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:11.978134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:11.978142 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:11.980133 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.980583 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.980584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.980585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.980668 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:11.980983 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:11.980984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:11.980985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:11.981218 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:11.981334 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:11.981716 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 07:00:11.981827 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:00:11.982072 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:11.969879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44887 (local address 127.24.181.66:37881)
0808 07:00:11.970214 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:11.970253 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:11.971658 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:11.971902 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:11.971906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:11.972213 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:11.972532 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:11.972537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.973467 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.973468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:11.974526 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:11.974528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:11.975095 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:11.975096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:11.975419 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:11.976151 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:11.976163 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:11.978022 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:11.980237 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:11.980245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:11.980247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:11.980498 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:11.980778 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:11.980778 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:11.980779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:11.980889 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:11.981330 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:11.981337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:11.981497 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:00:11.981651 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 07:00:11.981803 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":198,"threads_started":1}
I20260808 07:00:11.982779 26568 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44887
I20260808 07:00:11.982873 26568 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:11.983088 26568 heartbeater.cc:511] Master 127.24.181.126:44887 requested a full tablet report, sending...
I20260808 07:00:11.983549 26100 ts_manager.cc:194] Registered new tserver with Master: 4222b0a05b364500b191044ea5cab067 (127.24.181.66:40045)
I20260808 07:00:11.984177 26100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:37881
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:12.004485 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:41881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:12.127526 26577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:12.127759 26577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:12.127787 26577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:12.129316 26577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:12.129371 26577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:12.129415 26577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:12.132154 26577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:41881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:12.132434 26577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:12.132673 26577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:12.139074 26587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:12.139106 26584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:12.139264 26585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:12.139523 26577 server_base.cc:1034] running on GCE node
I20260808 07:00:12.139739 26577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:12.140008 26577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:12.141155 26577 hybrid_clock.cc:648] HybridClock initialized: now 1786172412141134 us; error 41 us; skew 500 ppm
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:12.144466 26577 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:12.144956 26577 webserver.cc:460] Webserver started at http://127.24.181.67:39755/ using document root <none> and password file <none>
I20260808 07:00:12.145187 26577 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:12.145260 26577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:12.145413 26577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:12.146622 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "e582cf3ee7fc4fd080c841c50e3a421c"
format_stamp: "Formatted at 2026-08-08 07:00:12 on dist-test-slave-nfb5"
server_key: "92631ba537d5dd52c19fac4a2cabd222"
server_key_iv: "b9d6709817982bc526be56007d29dd22"
server_key_version: "encryptionkey@0"
I20260808 07:00:12.147111 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "e582cf3ee7fc4fd080c841c50e3a421c"
format_stamp: "Formatted at 2026-08-08 07:00:12 on dist-test-slave-nfb5"
server_key: "92631ba537d5dd52c19fac4a2cabd222"
server_key_iv: "b9d6709817982bc526be56007d29dd22"
server_key_version: "encryptionkey@0"
I20260808 07:00:12.148784 26577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:12.149771 26599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:12.150007 26577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:12.150117 26577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "e582cf3ee7fc4fd080c841c50e3a421c"
format_stamp: "Formatted at 2026-08-08 07:00:12 on dist-test-slave-nfb5"
server_key: "92631ba537d5dd52c19fac4a2cabd222"
server_key_iv: "b9d6709817982bc526be56007d29dd22"
server_key_version: "encryptionkey@0"
I20260808 07:00:12.150194 26577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:12.166349 26577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:12.169032 26577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:12.169260 26577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:12.169677 26577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:12.170068 26577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:12.170128 26577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:12.170163 26577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:12.170197 26577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:12.186230 26577 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:45929
I20260808 07:00:12.186444 26758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:45929 every 8 connection(s)
I20260808 07:00:12.186761 26577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:12.191074 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 26577
I20260808 07:00:12.191200 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:12.191366 25301 external_mini_cluster.cc:1428] Setting key b849318f1dfff778ebb586600681f808
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:12.198202 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.187927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:42191 (local address 127.24.181.126:44887)
0808 07:00:12.187999 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.188001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.188380 (+ 379us) server_negotiation.cc:407] Connection header received
0808 07:00:12.189478 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.189479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.189493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.189543 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:12.190353 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.190955 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.192494 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.192698 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.192774 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.195435 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:12.195437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:12.195438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:12.195445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:12.196729 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.197149 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.197150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.197151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.197232 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.197495 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.197496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.197497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.197655 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:12.197738 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.198008 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.198102 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:00:12.198444 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.187893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44887 (local address 127.24.181.67:42191)
0808 07:00:12.188231 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.188270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.189357 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.189614 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.189617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.189935 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:12.190239 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.190244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.191037 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.191038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.192404 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.192407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.192795 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.192796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.192977 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.193615 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:12.193624 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:12.195323 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:12.196823 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:12.196830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:12.196833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:12.197059 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:12.197307 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:12.197308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:12.197308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:12.197417 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:12.197728 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:12.197736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:12.197873 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.197986 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.198142 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":192,"threads_started":1}
I20260808 07:00:12.199039 26760 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44887
I20260808 07:00:12.199144 26760 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:12.199334 26760 heartbeater.cc:511] Master 127.24.181.126:44887 requested a full tablet report, sending...
I20260808 07:00:12.199707 26100 ts_manager.cc:194] Registered new tserver with Master: e582cf3ee7fc4fd080c841c50e3a421c (127.24.181.67:45929)
I20260808 07:00:12.200119 26100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:42191
I20260808 07:00:12.203265 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:12.211485 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.205167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.24.181.126:44887)
0808 07:00:12.205264 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.205266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.205435 (+ 169us) server_negotiation.cc:407] Connection header received
0808 07:00:12.205511 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.205512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.205523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.205596 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:12.206232 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.206689 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.207491 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.207674 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.207779 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.208931 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:12.208933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:12.208933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:12.208940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:12.210210 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.210595 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.210596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.210596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.210683 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.210895 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.210896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.210896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.211077 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:12.211194 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.211310 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.211389 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 07:00:12.213910 26100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:12.215723 26100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:12.222837 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.217993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:40045 (local address 127.0.0.1:40934)
0808 07:00:12.218475 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.218523 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.218612 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.219323 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.219324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.219327 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:12.219456 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.219459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.220537 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.220538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.221667 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.221669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.222436 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.222437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.222592 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.222605 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.222701 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.222765 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":49,"threads_started":1}
I20260808 07:00:12.222882 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.218309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.24.181.66:40045)
0808 07:00:12.218874 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.218879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.218890 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:12.218992 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.219003 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.219019 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.219120 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:12.219551 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.220367 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.221755 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.222329 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.222431 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.222495 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.222722 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.222766 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":153,"threads_started":1}
I20260808 07:00:12.223551 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.217995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:45929 (local address 127.0.0.1:52168)
0808 07:00:12.218501 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.218534 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.218600 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.219243 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.219244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.219246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:12.219294 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.219295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.221178 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.221179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.222149 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.222152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.223322 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.223323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.223389 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.223393 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.223428 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.223477 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":235,"threads_started":1}
I20260808 07:00:12.224018 26473 tablet_service.cc:1467] Processing CreateTablet for tablet b5feba6b4b834f7a906b5e459b47f542 (DEFAULT_TABLE table=test-table [id=96aa613691714d18aa44bdb030932e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:12.224226 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.218163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.24.181.67:45929)
0808 07:00:12.218947 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.218952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.218963 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:12.219045 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.219047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.219066 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.219496 (+ 430us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:12.219671 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.220974 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.222252 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.223212 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.223913 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.223982 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.224008 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.224049 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":204,"threads_started":1}
I20260808 07:00:12.224454 26473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5feba6b4b834f7a906b5e459b47f542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:12.225158 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.218040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33413 (local address 127.0.0.1:33516)
0808 07:00:12.218574 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.218585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.218659 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.219007 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.219010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.219012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:12.219173 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.219176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.221067 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.221068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.223390 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.223392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.224891 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.224892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.224968 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.224974 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.225040 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.225083 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":940,"thread_start_us":373,"threads_started":1}
I20260808 07:00:12.225250 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.218298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.24.181.65:33413)
0808 07:00:12.218476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.218481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.218667 (+ 186us) server_negotiation.cc:407] Connection header received
0808 07:00:12.218781 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.218793 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.218811 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.218936 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:12.219274 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.220828 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.223487 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.224788 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.224891 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.224953 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.225055 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.225098 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260808 07:00:12.225175 26664 tablet_service.cc:1467] Processing CreateTablet for tablet b5feba6b4b834f7a906b5e459b47f542 (DEFAULT_TABLE table=test-table [id=96aa613691714d18aa44bdb030932e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:12.225685 26664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5feba6b4b834f7a906b5e459b47f542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:12.226342 26290 tablet_service.cc:1467] Processing CreateTablet for tablet b5feba6b4b834f7a906b5e459b47f542 (DEFAULT_TABLE table=test-table [id=96aa613691714d18aa44bdb030932e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:12.226866 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5feba6b4b834f7a906b5e459b47f542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:12.230777 26786 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Bootstrap starting.
I20260808 07:00:12.231541 26789 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Bootstrap starting.
I20260808 07:00:12.232887 26786 tablet_bootstrap.cc:654] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:12.233507 26789 tablet_bootstrap.cc:654] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:12.234654 26786 log.cc:824] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:12.234969 26789 log.cc:824] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:12.235210 26788 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: Bootstrap starting.
I20260808 07:00:12.235962 26789 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: No bootstrap required, opened a new log
I20260808 07:00:12.236065 26789 ts_tablet_manager.cc:1397] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 07:00:12.236050 26786 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: No bootstrap required, opened a new log
I20260808 07:00:12.236160 26786 ts_tablet_manager.cc:1397] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 07:00:12.237102 26788 tablet_bootstrap.cc:654] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:12.238754 26788 log.cc:824] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:12.239295 26789 raft_consensus.cc:348] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.239568 26789 raft_consensus.cc:374] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:12.239621 26789 raft_consensus.cc:729] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e582cf3ee7fc4fd080c841c50e3a421c, State: Initialized, Role: FOLLOWER
I20260808 07:00:12.239750 26789 consensus_queue.cc:260] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.240000 26788 tablet_bootstrap.cc:492] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: No bootstrap required, opened a new log
I20260808 07:00:12.240073 26788 ts_tablet_manager.cc:1397] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 07:00:12.240247 26789 ts_tablet_manager.cc:1428] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 07:00:12.240391 26760 heartbeater.cc:503] Master 127.24.181.126:44887 was elected leader, sending a full tablet report...
I20260808 07:00:12.241436 26786 raft_consensus.cc:348] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.241760 26786 raft_consensus.cc:374] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:12.241837 26786 raft_consensus.cc:729] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4222b0a05b364500b191044ea5cab067, State: Initialized, Role: FOLLOWER
I20260808 07:00:12.241961 26786 consensus_queue.cc:260] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.242447 26568 heartbeater.cc:503] Master 127.24.181.126:44887 was elected leader, sending a full tablet report...
I20260808 07:00:12.242530 26786 ts_tablet_manager.cc:1428] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 07:00:12.244436 26788 raft_consensus.cc:348] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.244562 26788 raft_consensus.cc:374] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:12.244586 26788 raft_consensus.cc:729] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c251ec276674cf2a999671fe8c766e1, State: Initialized, Role: FOLLOWER
I20260808 07:00:12.244664 26788 consensus_queue.cc:260] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.245043 26382 heartbeater.cc:503] Master 127.24.181.126:44887 was elected leader, sending a full tablet report...
I20260808 07:00:12.245029 26788 ts_tablet_manager.cc:1428] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 07:00:12.383347 26797 raft_consensus.cc:482] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:12.383534 26797 raft_consensus.cc:504] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.384089 26797 leader_election.cc:288] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e582cf3ee7fc4fd080c841c50e3a421c (127.24.181.67:45929), 4222b0a05b364500b191044ea5cab067 (127.24.181.66:40045)
I20260808 07:00:12.387660 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.384383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:51553 (local address 127.24.181.66:40045)
0808 07:00:12.384448 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.384450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.384582 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:00:12.384725 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.384727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.384740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.384814 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:12.385042 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.385581 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.386720 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.387217 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.387324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.387334 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.387549 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.387590 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:12.387704 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.384304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:40045 (local address 127.24.181.65:51553)
0808 07:00:12.384482 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.384496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.384633 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.384825 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.384826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.384828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:12.384949 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.384951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.385662 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.385663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.386529 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.386531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.387317 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.387318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.387462 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.387466 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.387542 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.387585 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260808 07:00:12.388137 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.384496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:55057 (local address 127.24.181.67:45929)
0808 07:00:12.384556 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.384558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.384706 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:00:12.384841 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.384842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.384852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.384931 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:12.385215 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.385731 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.387136 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.387620 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.387713 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.387724 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.388034 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.388076 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:00:12.388185 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.384306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:45929 (local address 127.24.181.65:55057)
0808 07:00:12.384628 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:12.384638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:12.384766 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:12.384934 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:12.384935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:12.384937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:12.385127 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.385129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.385813 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.385814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.387044 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:12.387047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.387738 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.387739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:12.387950 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.387953 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:12.388025 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:00:12.388063 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":3,"thread_start_us":209,"threads_started":1}
I20260808 07:00:12.388327 26504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5feba6b4b834f7a906b5e459b47f542" candidate_uuid: "8c251ec276674cf2a999671fe8c766e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4222b0a05b364500b191044ea5cab067" is_pre_election: true
I20260808 07:00:12.388521 26504 raft_consensus.cc:2454] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c251ec276674cf2a999671fe8c766e1 in term 0.
I20260808 07:00:12.388590 26697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5feba6b4b834f7a906b5e459b47f542" candidate_uuid: "8c251ec276674cf2a999671fe8c766e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" is_pre_election: true
I20260808 07:00:12.388784 26697 raft_consensus.cc:2454] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c251ec276674cf2a999671fe8c766e1 in term 0.
I20260808 07:00:12.388923 26227 leader_election.cc:302] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4222b0a05b364500b191044ea5cab067, 8c251ec276674cf2a999671fe8c766e1; no voters:
I20260808 07:00:12.389119 26797 raft_consensus.cc:2781] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:12.389166 26797 raft_consensus.cc:482] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:12.389183 26797 raft_consensus.cc:3037] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:12.390327 26797 raft_consensus.cc:504] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.390487 26797 leader_election.cc:288] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [CANDIDATE]: Term 1 election: Requested vote from peers e582cf3ee7fc4fd080c841c50e3a421c (127.24.181.67:45929), 4222b0a05b364500b191044ea5cab067 (127.24.181.66:40045)
I20260808 07:00:12.390741 26504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5feba6b4b834f7a906b5e459b47f542" candidate_uuid: "8c251ec276674cf2a999671fe8c766e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4222b0a05b364500b191044ea5cab067"
I20260808 07:00:12.390779 26697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5feba6b4b834f7a906b5e459b47f542" candidate_uuid: "8c251ec276674cf2a999671fe8c766e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e582cf3ee7fc4fd080c841c50e3a421c"
I20260808 07:00:12.390834 26504 raft_consensus.cc:3037] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:12.390890 26697 raft_consensus.cc:3037] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:12.392154 26504 raft_consensus.cc:2454] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c251ec276674cf2a999671fe8c766e1 in term 1.
I20260808 07:00:12.392356 26227 leader_election.cc:302] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4222b0a05b364500b191044ea5cab067, 8c251ec276674cf2a999671fe8c766e1; no voters:
I20260808 07:00:12.392379 26697 raft_consensus.cc:2454] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c251ec276674cf2a999671fe8c766e1 in term 1.
I20260808 07:00:12.392449 26797 raft_consensus.cc:2781] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:12.392623 26797 raft_consensus.cc:686] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8c251ec276674cf2a999671fe8c766e1, State: Running, Role: LEADER
I20260808 07:00:12.392747 26797 consensus_queue.cc:237] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } }
I20260808 07:00:12.393752 26100 catalog_manager.cc:5495] T b5feba6b4b834f7a906b5e459b47f542 P 8c251ec276674cf2a999671fe8c766e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c251ec276674cf2a999671fe8c766e1 (127.24.181.65). New cstate: current_term: 1 leader_uuid: "8c251ec276674cf2a999671fe8c766e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e582cf3ee7fc4fd080c841c50e3a421c" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4222b0a05b364500b191044ea5cab067" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c251ec276674cf2a999671fe8c766e1" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33413 } health_report { overall_health: HEALTHY } } }
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:12.407935 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.400035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.24.181.65:33413)
0808 07:00:12.400093 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.400095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.400255 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:00:12.400330 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.400331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.400341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.400396 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:12.401093 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.401556 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.402371 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.402568 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.402643 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.405224 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:12.405232 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:12.405232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:12.405239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:12.406979 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.407303 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.407306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.407306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.407361 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.407530 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.407530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.407531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.407652 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:12.407785 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.407814 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.407874 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:12.433359 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.426117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.24.181.65:33413)
0808 07:00:12.426204 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.426207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.426271 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:00:12.426324 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.426325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.426334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.426385 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:12.427066 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.427545 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.428308 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.428492 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.428543 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.430889 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:12.430890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:12.430890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:12.430896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:12.432392 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.432696 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.432697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.432698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.432749 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.432910 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.432911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.432911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.432997 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:12.433089 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.433195 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.433251 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 07:00:12.437752 26763 tablet.cc:2369] T b5feba6b4b834f7a906b5e459b47f542 P e582cf3ee7fc4fd080c841c50e3a421c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:12.458060 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:12.451243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.24.181.65:33413)
0808 07:00:12.451345 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:12.451347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:12.451403 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:00:12.451448 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:12.451448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:12.451458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:12.451507 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:12.452058 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.452514 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.453225 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:12.453469 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:12.453549 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:12.455906 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:12.455908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:12.455908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:12.455914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:12.457122 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.457457 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.457458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.457458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.457510 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:12.457675 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:12.457676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:12.457676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:12.457761 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:12.457843 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:12.457915 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:00:12.457971 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 07:00:12.459022 26262 tablet_service.cc:2908] Rejecting scan request for tablet b5feba6b4b834f7a906b5e459b47f542: Uninitialized: clean time has not yet been initialized
I20260808 07:00:12.459985 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 26196
W20260808 07:00:12.470016 26572 tablet.cc:2369] T b5feba6b4b834f7a906b5e459b47f542 P 4222b0a05b364500b191044ea5cab067: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:12.478897 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 26387
I20260808 07:00:12.489246 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 26577
I20260808 07:00:12.501829 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 26055
2026-08-08T07:00:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2486 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26821](info): set up 2 sockets
Aug 08 07:00:12 dist-test-slave-nfb5 krb5kdc[26821](info): commencing operation
krb5kdc: starting...
W20260808 07:00:13.726172 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:13 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:13.845844 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36053
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:13.928547 26838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:13.928782 26838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:13.928819 26838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:13.929963 26838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:13.930030 26838 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:00:13.930051 26838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:13.930074 26838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:13.930106 26838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:13.930128 26838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:13.931651 26838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:36053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:13.931932 26838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:13.932171 26838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:13.937041 26845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:13.937042 26849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:13.937088 26847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:13.937309 26838 server_base.cc:1034] running on GCE node
I20260808 07:00:13.937568 26838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:13.937855 26838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:13.938974 26838 hybrid_clock.cc:648] HybridClock initialized: now 1786172413938968 us; error 32 us; skew 500 ppm
Aug 08 07:00:13 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:13.944608 26838 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:13.945039 26838 webserver.cc:460] Webserver started at http://127.24.181.126:41877/ using document root <none> and password file <none>
I20260808 07:00:13.945271 26838 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:13.945335 26838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:13.945484 26838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:13.946482 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "7b5c8aa240b14ad59bf963d738aae062"
format_stamp: "Formatted at 2026-08-08 07:00:13 on dist-test-slave-nfb5"
server_key: "745fb378b341bf6193295d1198f22f19"
server_key_iv: "0fb4f82b096f83117cde971d68822534"
server_key_version: "encryptionkey@0"
I20260808 07:00:13.946866 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "7b5c8aa240b14ad59bf963d738aae062"
format_stamp: "Formatted at 2026-08-08 07:00:13 on dist-test-slave-nfb5"
server_key: "745fb378b341bf6193295d1198f22f19"
server_key_iv: "0fb4f82b096f83117cde971d68822534"
server_key_version: "encryptionkey@0"
I20260808 07:00:13.948305 26838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:13.949033 26861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:13.949221 26838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:13.949318 26838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "7b5c8aa240b14ad59bf963d738aae062"
format_stamp: "Formatted at 2026-08-08 07:00:13 on dist-test-slave-nfb5"
server_key: "745fb378b341bf6193295d1198f22f19"
server_key_iv: "0fb4f82b096f83117cde971d68822534"
server_key_version: "encryptionkey@0"
I20260808 07:00:13.949400 26838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:13.978176 26838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:13.980142 26838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:13.980345 26838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:13.985397 26838 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:36053
I20260808 07:00:13.985455 26957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:36053 every 8 connection(s)
I20260808 07:00:13.985854 26838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:13.986474 26958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:13.988811 26958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Bootstrap starting.
I20260808 07:00:13.989408 26958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:13.989718 26958 log.cc:824] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:13.990411 26958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: No bootstrap required, opened a new log
I20260808 07:00:13.990759 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 26838
I20260808 07:00:13.990842 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:13.991034 25301 external_mini_cluster.cc:1428] Setting key 5e759952996b954bb903773bb2d80533
I20260808 07:00:13.993222 26958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } }
I20260808 07:00:13.993396 26958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:13.993433 26958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b5c8aa240b14ad59bf963d738aae062, State: Initialized, Role: FOLLOWER
I20260808 07:00:13.993592 26958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } }
I20260808 07:00:13.993656 26958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:13.993678 26958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:13.993758 26958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:13.994457 26958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } }
I20260808 07:00:13.994565 26958 leader_election.cc:302] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b5c8aa240b14ad59bf963d738aae062; no voters:
I20260808 07:00:13.994740 26958 leader_election.cc:288] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:13.994807 26967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:13.995002 26958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:13.995049 26967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [term 1 LEADER]: Becoming Leader. State: Replica: 7b5c8aa240b14ad59bf963d738aae062, State: Running, Role: LEADER
I20260808 07:00:13.995249 26967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } }
I20260808 07:00:13.995683 26970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b5c8aa240b14ad59bf963d738aae062. Latest consensus state: current_term: 1 leader_uuid: "7b5c8aa240b14ad59bf963d738aae062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } } }
I20260808 07:00:13.995925 26970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:13.995671 26969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b5c8aa240b14ad59bf963d738aae062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5c8aa240b14ad59bf963d738aae062" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36053 } } }
I20260808 07:00:13.996153 26987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:13.996208 26969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:13.996677 26987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:00:13 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.000001 26987 catalog_manager.cc:1347] Generated new cluster ID: 8dc83999adf146aea3038942b7dd0928
I20260808 07:00:14.000053 26987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:14.005455 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:13.991552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.24.181.126:36053)
0808 07:00:13.991787 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:13.991796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:13.991830 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:00:13.993187 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:13.993195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:13.993265 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:13.993416 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:13.994181 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:13.995619 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:13.996753 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:13.997351 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:13.997463 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.000103 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.000107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.000107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.000119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.002253 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.002686 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.002689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.002689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.003622 (+ 933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.003883 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.003884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.003884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.004192 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.004997 (+ 805us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.005072 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.005246 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260808 07:00:14.018452 26987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:14.019160 26987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:14.027197 26987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Generated new TSK 0
I20260808 07:00:14.027400 26987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:14.059720 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36053
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:14.142513 27006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:14.142714 27006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:14.142740 27006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:14.144563 27006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:14.144619 27006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:14.144651 27006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:14.144739 27006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:14.146896 27006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:14.147186 27006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:14.147444 27006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:14.152501 27015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:14.152515 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:14.152673 27006 server_base.cc:1034] running on GCE node
W20260808 07:00:14.152603 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:14.152966 27006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:14.153244 27006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:14.154405 27006 hybrid_clock.cc:648] HybridClock initialized: now 1786172414154391 us; error 39 us; skew 500 ppm
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:14.156998 27006 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:14.157441 27006 webserver.cc:460] Webserver started at http://127.24.181.65:45431/ using document root <none> and password file <none>
I20260808 07:00:14.157634 27006 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:14.157675 27006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:14.157773 27006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:14.158805 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "1706e26b769fc2ae8bc382f45861539b"
server_key_iv: "b1c8b9fc3e288f3c697baa6e05911b89"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.159194 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "1706e26b769fc2ae8bc382f45861539b"
server_key_iv: "b1c8b9fc3e288f3c697baa6e05911b89"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.160530 27006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:14.161262 27026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.161453 27006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:14.161562 27006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "1706e26b769fc2ae8bc382f45861539b"
server_key_iv: "b1c8b9fc3e288f3c697baa6e05911b89"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.161638 27006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:14.186841 27006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:14.188776 27006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:14.188894 27006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:14.189368 27006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:14.189441 27006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.189483 27006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:14.189533 27006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.197237 27006 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:46397
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.197320 27188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:46397 every 8 connection(s)
I20260808 07:00:14.197611 27006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:14.200882 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.189981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:50911 (local address 127.24.181.126:36053)
0808 07:00:14.190062 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.190065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.190315 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:00:14.191079 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.191080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.191093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.191175 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.192134 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.193001 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.194651 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.195015 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.195081 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.197811 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.197813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.197813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.197830 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.199411 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.199842 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.199844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.199844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.199909 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.200173 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.200174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.200175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.200298 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.200377 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.200665 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.200775 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:00:14.201006 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.189943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.65:50911)
0808 07:00:14.190153 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.190215 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.190964 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.191274 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.191279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.191702 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.192008 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.192015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.193106 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.193108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.194544 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.194547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.195098 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.195099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.195295 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.195861 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.195874 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.197727 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.199518 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.199526 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.199529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.199755 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.199982 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.199983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.199984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.200101 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.200368 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.200375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.200514 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.200619 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.200787 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260808 07:00:14.201556 27189 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36053
I20260808 07:00:14.201665 27189 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:14.201862 27189 heartbeater.cc:511] Master 127.24.181.126:36053 requested a full tablet report, sending...
I20260808 07:00:14.202419 26891 ts_manager.cc:194] Registered new tserver with Master: df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397)
I20260808 07:00:14.203379 26891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:50911
I20260808 07:00:14.205145 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 27006
I20260808 07:00:14.205257 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:14.205459 25301 external_mini_cluster.cc:1428] Setting key 3d2cc8415cb5e884a1e9a8de724b79b1
I20260808 07:00:14.208482 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.202734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:43101 (local address 127.24.181.126:36053)
0808 07:00:14.202774 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.202776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.202874 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:00:14.202978 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.202979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.202988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.203042 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.203385 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.203990 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.204691 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.204872 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.204921 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.205911 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.205911 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.205912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.205917 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.207357 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.207718 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.207719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.207720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.207777 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.207987 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.207988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.207989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.208089 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.208164 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.208315 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.208395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:00:14.208498 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.202666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.65:43101)
0808 07:00:14.202818 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.202829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.202889 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.203050 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.203051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.203255 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.203315 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.203317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.204096 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.204097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.204619 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.204620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.204952 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.204952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.205008 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.205445 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.205450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.205838 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.207474 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.207475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.207475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.207646 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.207858 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.207859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.207859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.207917 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.208157 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.208158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.208203 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.208290 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.208395 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:14.233474 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36053
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:14.316970 27210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:14.317169 27210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:14.317193 27210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:14.318428 27210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:14.318471 27210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:14.318493 27210 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:14.318543 27210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:14.319995 27210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:14.320240 27210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:14.320467 27210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:14.325480 27220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:14.325486 27217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:14.325479 27216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:14.325861 27210 server_base.cc:1034] running on GCE node
I20260808 07:00:14.326076 27210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:14.326319 27210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:14.327493 27210 hybrid_clock.cc:648] HybridClock initialized: now 1786172414327487 us; error 37 us; skew 500 ppm
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:14.329994 27210 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:14.330475 27210 webserver.cc:460] Webserver started at http://127.24.181.66:35373/ using document root <none> and password file <none>
I20260808 07:00:14.330710 27210 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:14.330781 27210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:14.330981 27210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:14.332052 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "85bb6991fe2d46dba717aba86e5841a5"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "64e12874fa78f927f0b399be0efdaee5"
server_key_iv: "14847532018e5b8e8fdb6bebaf08514a"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.332422 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "85bb6991fe2d46dba717aba86e5841a5"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "64e12874fa78f927f0b399be0efdaee5"
server_key_iv: "14847532018e5b8e8fdb6bebaf08514a"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.333773 27210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:14.334493 27231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.334669 27210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:14.334772 27210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "85bb6991fe2d46dba717aba86e5841a5"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "64e12874fa78f927f0b399be0efdaee5"
server_key_iv: "14847532018e5b8e8fdb6bebaf08514a"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.334857 27210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:14.359166 27210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:14.361020 27210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:14.361192 27210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:14.361716 27210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:14.361783 27210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.361828 27210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:14.361873 27210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.368955 27210 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:45329
I20260808 07:00:14.369113 27398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:45329 every 8 connection(s)
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.369449 27210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:14.372366 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.362286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:54341 (local address 127.24.181.126:36053)
0808 07:00:14.362366 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.362369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.362548 (+ 179us) server_negotiation.cc:407] Connection header received
0808 07:00:14.363132 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.363133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.363147 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.363204 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.364101 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.364988 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.366619 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.366971 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.367068 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.369813 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.369816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.369816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.369824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.371136 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.371515 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.371516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.371516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.371567 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.371810 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.371811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.371811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.371957 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.372027 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.372236 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.372302 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:00:14.372545 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.362243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.66:54341)
0808 07:00:14.362436 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.362476 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.363025 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.363274 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.363278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.363685 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.363981 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.363987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.365119 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.365120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.366513 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.366516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.367073 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.367074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.367175 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.367694 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.367720 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.369690 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.371207 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.371213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.371215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.371446 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.371626 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.371627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.371627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.371724 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.372027 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.372032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.372138 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.372220 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.372349 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260808 07:00:14.373126 27399 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36053
I20260808 07:00:14.373215 27399 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:14.373404 27399 heartbeater.cc:511] Master 127.24.181.126:36053 requested a full tablet report, sending...
I20260808 07:00:14.373848 26892 ts_manager.cc:194] Registered new tserver with Master: 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.374336 26892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:54341
I20260808 07:00:14.379205 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 27210
I20260808 07:00:14.379312 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:14.379546 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.374268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.66:60849)
0808 07:00:14.374430 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.374442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.374525 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.374677 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.374678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.374895 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.374973 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.374975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.375608 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.375609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.376087 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.376089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.376433 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.376434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.376489 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.376939 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.376944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.377224 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.378516 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.378517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.378517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.378674 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.378862 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.378862 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.378863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.378953 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.379228 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.379230 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.379280 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.379358 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.379464 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260808 07:00:14.379557 25301 external_mini_cluster.cc:1428] Setting key 4ecb025ed052d30dda99b39424d784cf
I20260808 07:00:14.379590 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.374367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60849 (local address 127.24.181.126:36053)
0808 07:00:14.374418 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.374419 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.374500 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:00:14.374576 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.374577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.374589 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.374664 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.375049 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.375516 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.376175 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.376364 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.376421 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.377302 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.377303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.377303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.377308 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.378442 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.378754 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.378755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.378755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.378803 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.379012 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.379013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.379013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.379141 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.379229 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.379409 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.379496 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:14.405162 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36053
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:14.488571 27411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:14.488827 27411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:14.488868 27411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:14.490026 27411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:14.490093 27411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:14.490128 27411 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:14.490170 27411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:14.491628 27411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:14.491918 27411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:14.492156 27411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:14.497525 27417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:14.497596 27418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:14.497774 27411 server_base.cc:1034] running on GCE node
W20260808 07:00:14.497862 27421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:14.498107 27411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:14.498328 27411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:14.499476 27411 hybrid_clock.cc:648] HybridClock initialized: now 1786172414499470 us; error 38 us; skew 500 ppm
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:14.502228 27411 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:14.502748 27411 webserver.cc:460] Webserver started at http://127.24.181.67:42319/ using document root <none> and password file <none>
I20260808 07:00:14.503082 27411 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:14.503155 27411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:14.503324 27411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:14.504429 27411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "f406b2dfd93d489d9591bd2abf0f4292"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "0f19753b0910289ba1da0330c6a7c692"
server_key_iv: "9ac4e635e1112cb77c46e3aab5c1dffb"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.504827 27411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "f406b2dfd93d489d9591bd2abf0f4292"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "0f19753b0910289ba1da0330c6a7c692"
server_key_iv: "9ac4e635e1112cb77c46e3aab5c1dffb"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.506228 27411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:14.507046 27432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.507225 27411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:14.507349 27411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "f406b2dfd93d489d9591bd2abf0f4292"
format_stamp: "Formatted at 2026-08-08 07:00:14 on dist-test-slave-nfb5"
server_key: "0f19753b0910289ba1da0330c6a7c692"
server_key_iv: "9ac4e635e1112cb77c46e3aab5c1dffb"
server_key_version: "encryptionkey@0"
I20260808 07:00:14.507432 27411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:14.522763 27411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:14.525255 27411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:14.525427 27411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:14.526098 27411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:14.526149 27411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:14.526206 27411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:14.526262 27411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.533921 27411 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:33895
I20260808 07:00:14.534286 27411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:14.534384 27595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:33895 every 8 connection(s)
I20260808 07:00:14.536577 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.526555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:51011 (local address 127.24.181.126:36053)
0808 07:00:14.526631 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.526634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.526857 (+ 223us) server_negotiation.cc:407] Connection header received
0808 07:00:14.527517 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.527519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.527535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.527618 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.528306 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.529232 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.530383 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.530735 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.530825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.533699 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.533701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.533701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.533710 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.535404 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.535795 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.535796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.535796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.535849 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.536064 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.536064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.536065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.536167 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.536217 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.536451 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.536516 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:14.536744 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.526546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.67:51011)
0808 07:00:14.526711 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.526752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.527411 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.527728 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.527732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.528009 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.528203 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.528208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.529342 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.529343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.530272 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.530274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.530894 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.530895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.531060 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.531596 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.531604 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.533592 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.535479 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.535484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.535485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.535702 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.535909 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.535910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.535910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.536008 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.536222 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.536227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.536340 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.536422 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.536550 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260808 07:00:14.537249 27598 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36053
I20260808 07:00:14.537338 27598 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:14.537557 27598 heartbeater.cc:511] Master 127.24.181.126:36053 requested a full tablet report, sending...
I20260808 07:00:14.537956 26891 ts_manager.cc:194] Registered new tserver with Master: f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67:33895)
I20260808 07:00:14.538466 26891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:51011
I20260808 07:00:14.540216 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 27411
I20260808 07:00:14.540333 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:14.540506 25301 external_mini_cluster.cc:1428] Setting key 25335f11233a02b18bf0291aec8decb8
I20260808 07:00:14.541484 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:14.544557 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.538338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.24.181.67:57707)
0808 07:00:14.538465 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.538476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.538523 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.538748 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.538749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.538985 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.539058 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.539061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.539699 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.539700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.540191 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.540192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.540770 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.540771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.540855 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.541346 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.541352 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.541883 (+ 531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.543531 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.543533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.543533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.543765 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.543980 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.543981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.543981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.544035 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.544315 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.544317 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.544354 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.544406 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.544485 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260808 07:00:14.544638 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.538424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:57707 (local address 127.24.181.126:36053)
0808 07:00:14.538630 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.538632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.538639 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:00:14.538656 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.538656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.538664 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.538749 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.539129 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.539587 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.540287 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.540587 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.540671 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.541996 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.541998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.541998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.542005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.543433 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.543841 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.543842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.543843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.543910 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.544099 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.544100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.544100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.544251 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.544325 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.544434 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.544512 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.565922 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.559102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.24.181.126:36053)
0808 07:00:14.559171 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.559173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.559190 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:00:14.559215 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.559216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.559225 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.559277 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.559836 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.560311 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.561030 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.561229 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.561285 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.563494 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.563496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.563496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.563503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.565017 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.565324 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.565324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.565325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.565376 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.565537 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.565537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.565538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.565635 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.565732 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.565796 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.565856 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:00:14.567643 26891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:14.569233 26891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:14.576196 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:33895 (local address 127.0.0.1:55668)
0808 07:00:14.571679 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.571688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.571753 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.572355 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.572358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.572360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:14.572450 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.572453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.573488 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.573489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.574857 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.574860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.575925 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.575926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.575978 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.575984 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.576061 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.576099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260808 07:00:14.576239 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.24.181.67:33895)
0808 07:00:14.572113 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.572119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.572130 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:14.572170 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.572172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.572187 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.572274 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:14.572536 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.573406 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.574981 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.575737 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.575820 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.575893 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.576097 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.576143 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260808 07:00:14.576196 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:46397 (local address 127.0.0.1:35734)
0808 07:00:14.571637 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.571685 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.571753 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.572360 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.572361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.572363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:14.572435 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.572438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.573792 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.573793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.574859 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.574860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.575866 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.575867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.575968 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.575984 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.576033 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.576094 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260808 07:00:14.576813 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.24.181.65:46397)
0808 07:00:14.572146 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.572150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.572162 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:00:14.572224 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.572226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.572242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.572518 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:14.572583 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.573718 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.574978 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.575760 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.576404 (+ 644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.576490 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.576580 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.576691 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":101,"threads_started":1}
I20260808 07:00:14.577124 27504 tablet_service.cc:1467] Processing CreateTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 (DEFAULT_TABLE table=test-table [id=73f588a979f44c9b8df2c5a05208071d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.577476 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45329 (local address 127.0.0.1:60106)
0808 07:00:14.571958 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.571969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.572023 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.572393 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.572394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.572395 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:14.572436 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.572439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.574985 (+ 2546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.574986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.575741 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.575743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.577228 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.577229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.577292 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.577296 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.577357 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.577397 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260808 07:00:14.577507 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd634bb328f14e6f9f984e5414a4ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.577455 27101 tablet_service.cc:1467] Processing CreateTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 (DEFAULT_TABLE table=test-table [id=73f588a979f44c9b8df2c5a05208071d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.577814 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd634bb328f14e6f9f984e5414a4ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.577828 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.571964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.24.181.66:45329)
0808 07:00:14.572158 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.572162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.572172 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:14.572224 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.572226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.572242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.572325 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:14.572522 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.573699 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.575858 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.577133 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.577216 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.577271 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.577657 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.577712 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260808 07:00:14.578450 27303 tablet_service.cc:1467] Processing CreateTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 (DEFAULT_TABLE table=test-table [id=73f588a979f44c9b8df2c5a05208071d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.578809 27303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd634bb328f14e6f9f984e5414a4ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.580390 27633 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Bootstrap starting.
I20260808 07:00:14.581218 27633 tablet_bootstrap.cc:654] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.581585 27633 log.cc:824] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:14.581750 27635 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Bootstrap starting.
I20260808 07:00:14.582463 27633 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: No bootstrap required, opened a new log
I20260808 07:00:14.582468 27635 tablet_bootstrap.cc:654] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.582543 27633 ts_tablet_manager.cc:1397] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:14.582805 27637 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Bootstrap starting.
I20260808 07:00:14.582827 27635 log.cc:824] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:14.583551 27637 tablet_bootstrap.cc:654] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.583699 27635 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: No bootstrap required, opened a new log
I20260808 07:00:14.583774 27635 ts_tablet_manager.cc:1397] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:14.584549 27637 log.cc:824] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:14.584649 27633 raft_consensus.cc:348] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.584823 27633 raft_consensus.cc:374] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.584861 27633 raft_consensus.cc:729] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.584982 27633 consensus_queue.cc:260] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.585445 27633 ts_tablet_manager.cc:1428] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:14.585641 27598 heartbeater.cc:503] Master 127.24.181.126:36053 was elected leader, sending a full tablet report...
I20260808 07:00:14.585732 27637 tablet_bootstrap.cc:492] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: No bootstrap required, opened a new log
I20260808 07:00:14.585796 27637 ts_tablet_manager.cc:1397] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:00:14.586328 27635 raft_consensus.cc:348] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.586483 27635 raft_consensus.cc:374] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.586519 27635 raft_consensus.cc:729] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2d4f0406c8473db7d36e4d081b8b6a, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.586648 27635 consensus_queue.cc:260] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.587078 27635 ts_tablet_manager.cc:1428] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:14.587239 27189 heartbeater.cc:503] Master 127.24.181.126:36053 was elected leader, sending a full tablet report...
I20260808 07:00:14.589516 27637 raft_consensus.cc:348] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.589671 27637 raft_consensus.cc:374] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.589720 27637 raft_consensus.cc:729] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85bb6991fe2d46dba717aba86e5841a5, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.589843 27637 consensus_queue.cc:260] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.590149 27637 ts_tablet_manager.cc:1428] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260808 07:00:14.590204 27399 heartbeater.cc:503] Master 127.24.181.126:36053 was elected leader, sending a full tablet report...
I20260808 07:00:14.611405 27643 raft_consensus.cc:482] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:14.611529 27643 raft_consensus.cc:504] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.611869 27643 leader_election.cc:288] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.616225 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.612407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:47189 (local address 127.24.181.66:45329)
0808 07:00:14.612461 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.612463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.612669 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:00:14.612751 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.612753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.612762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.612814 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:14.612965 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.613609 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615026 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.615613 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615718 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.615728 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.616057 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.616130 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:00:14.616490 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.612358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45329 (local address 127.24.181.67:47189)
0808 07:00:14.612605 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.612615 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.612676 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.612827 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.612827 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.612829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:14.612896 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.612897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.613692 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.613693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.614944 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.614946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.615768 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.615864 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.615873 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.616344 (+ 471us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.616405 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":182,"threads_started":1}
I20260808 07:00:14.616707 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.612171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:45719 (local address 127.24.181.65:46397)
0808 07:00:14.612253 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.612255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.612281 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:00:14.612380 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.612381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.612391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.612453 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:14.612645 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.613409 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615026 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.615769 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615841 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.615850 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.616539 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.616612 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:00:14.616916 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.612136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:46397 (local address 127.24.181.67:45719)
0808 07:00:14.612219 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.612233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.612301 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.612495 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.612496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.612498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:14.612570 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.612571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.613495 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.613496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.614901 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.614904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.615922 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.615924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.616318 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.616352 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.616801 (+ 449us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.616856 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260808 07:00:14.616894 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd634bb328f14e6f9f984e5414a4ea56" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5" is_pre_election: true
I20260808 07:00:14.617069 27332 raft_consensus.cc:2454] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.617327 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd634bb328f14e6f9f984e5414a4ea56" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" is_pre_election: true
I20260808 07:00:14.617374 27434 leader_election.cc:302] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85bb6991fe2d46dba717aba86e5841a5, f406b2dfd93d489d9591bd2abf0f4292; no voters:
I20260808 07:00:14.617497 27128 raft_consensus.cc:2454] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.617511 27643 raft_consensus.cc:2781] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:14.617573 27643 raft_consensus.cc:482] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:14.617630 27643 raft_consensus.cc:3037] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.618628 27643 raft_consensus.cc:504] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.618769 27643 leader_election.cc:288] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Requested vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.618952 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd634bb328f14e6f9f984e5414a4ea56" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
I20260808 07:00:14.618992 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd634bb328f14e6f9f984e5414a4ea56" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5"
I20260808 07:00:14.619072 27128 raft_consensus.cc:3037] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.619074 27332 raft_consensus.cc:3037] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.619869 27128 raft_consensus.cc:2454] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:14.620064 27434 leader_election.cc:302] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df2d4f0406c8473db7d36e4d081b8b6a, f406b2dfd93d489d9591bd2abf0f4292; no voters:
W20260808 07:00:14.620154 27400 tablet.cc:2369] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:14.620361 27643 raft_consensus.cc:2781] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:14.620582 27643 raft_consensus.cc:686] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 LEADER]: Becoming Leader. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Running, Role: LEADER
I20260808 07:00:14.620679 27643 consensus_queue.cc:237] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.621444 27332 raft_consensus.cc:2454] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:14.621728 26891 catalog_manager.cc:5495] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 reported cstate change: term changed from 0 to 1, leader changed from <none> to f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:14.637403 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.634674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.24.181.67:33895)
0808 07:00:14.634721 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.634722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.634752 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:00:14.634841 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.634841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.634858 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.634972 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:14.635117 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.635592 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.636345 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.636520 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.636570 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.636760 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:14.636851 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:14.636907 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.637024 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.637050 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:00:14.639760 27128 raft_consensus.cc:1261] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:14.639797 27332 raft_consensus.cc:1261] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:14.640182 27643 consensus_queue.cc:1035] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:14.640379 27648 consensus_queue.cc:1035] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:14.643481 27653 mvcc.cc:205] Tried to move back new op lower bound from 7316162210361577472 to 7316162210287091712. Current Snapshot: MvccSnapshot[applied={T|T < 7316162210361577472}]
I20260808 07:00:14.643677 27656 mvcc.cc:205] Tried to move back new op lower bound from 7316162210361577472 to 7316162210287091712. Current Snapshot: MvccSnapshot[applied={T|T < 7316162210361577472}]
I20260808 07:00:14.648649 26892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:14.648806 26892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:14.649632 26892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Sending DeleteTablet for 3 replicas of tablet bd634bb328f14e6f9f984e5414a4ea56
I20260808 07:00:14.650065 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:14 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55668
I20260808 07:00:14.650089 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:14 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:35734
I20260808 07:00:14.650415 27303 tablet_service.cc:1514] Processing DeleteTablet for tablet bd634bb328f14e6f9f984e5414a4ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:14 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:60106
I20260808 07:00:14.650348 26892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:00:14.650496 27668 tablet_replica.cc:330] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: stopping tablet replica
W20260808 07:00:14.650537 26892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:14.650595 27668 raft_consensus.cc:2229] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:14.650708 27668 raft_consensus.cc:2258] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:14.651400 27668 ts_tablet_manager.cc:1905] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:14.652010 27672 tablet_replica.cc:330] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: stopping tablet replica
I20260808 07:00:14.652340 27101 tablet_service.cc:1467] Processing CreateTablet for tablet 44fc54963e334456acb95c313fe6f3c4 (DEFAULT_TABLE table=test-table [id=11074550f21c40dfab435b64d1703299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.652509 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fc54963e334456acb95c313fe6f3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.652904 27672 raft_consensus.cc:2229] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:14.653538 27669 tablet_replica.cc:330] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: stopping tablet replica
I20260808 07:00:14.654034 27672 raft_consensus.cc:2258] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:14.654259 27669 raft_consensus.cc:2229] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:14.654333 27668 ts_tablet_manager.cc:1918] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:14.654408 27668 log.cc:1196] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/bd634bb328f14e6f9f984e5414a4ea56
I20260808 07:00:14.654650 27669 raft_consensus.cc:2258] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:14.654874 27303 tablet_service.cc:1467] Processing CreateTablet for tablet 44fc54963e334456acb95c313fe6f3c4 (DEFAULT_TABLE table=test-table [id=11074550f21c40dfab435b64d1703299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.654888 27668 ts_tablet_manager.cc:1939] T bd634bb328f14e6f9f984e5414a4ea56 P f406b2dfd93d489d9591bd2abf0f4292: Deleting consensus metadata
I20260808 07:00:14.655040 27303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fc54963e334456acb95c313fe6f3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.655859 27669 ts_tablet_manager.cc:1905] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:14.655864 27672 ts_tablet_manager.cc:1905] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:14.657148 27637 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: Bootstrap starting.
I20260808 07:00:14.657197 27635 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Bootstrap starting.
I20260808 07:00:14.658160 27672 ts_tablet_manager.cc:1918] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:14.658239 27672 log.cc:1196] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/bd634bb328f14e6f9f984e5414a4ea56
I20260808 07:00:14.658280 27637 tablet_bootstrap.cc:654] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.658582 27635 tablet_bootstrap.cc:654] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.658615 27672 ts_tablet_manager.cc:1939] T bd634bb328f14e6f9f984e5414a4ea56 P 85bb6991fe2d46dba717aba86e5841a5: Deleting consensus metadata
I20260808 07:00:14.659696 26863 catalog_manager.cc:4841] TS f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67:33895): tablet bd634bb328f14e6f9f984e5414a4ea56 (table test-table [id=73f588a979f44c9b8df2c5a05208071d]) successfully deleted
I20260808 07:00:14.660035 27504 tablet_service.cc:1467] Processing CreateTablet for tablet 44fc54963e334456acb95c313fe6f3c4 (DEFAULT_TABLE table=test-table [id=11074550f21c40dfab435b64d1703299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:14.660153 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fc54963e334456acb95c313fe6f3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.660192 26865 catalog_manager.cc:4841] TS 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329): tablet bd634bb328f14e6f9f984e5414a4ea56 (table test-table [id=73f588a979f44c9b8df2c5a05208071d]) successfully deleted
I20260808 07:00:14.660656 27635 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: No bootstrap required, opened a new log
I20260808 07:00:14.660707 27635 ts_tablet_manager.cc:1397] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 07:00:14.660916 27669 ts_tablet_manager.cc:1918] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:14.660985 27669 log.cc:1196] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/bd634bb328f14e6f9f984e5414a4ea56
I20260808 07:00:14.661168 27635 raft_consensus.cc:348] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.661262 27635 raft_consensus.cc:374] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.661304 27635 raft_consensus.cc:729] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2d4f0406c8473db7d36e4d081b8b6a, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.661387 27635 consensus_queue.cc:260] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.661496 27635 ts_tablet_manager.cc:1428] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:14.661861 27633 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Bootstrap starting.
I20260808 07:00:14.661875 27637 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: No bootstrap required, opened a new log
I20260808 07:00:14.661926 27637 ts_tablet_manager.cc:1397] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 07:00:14.662309 27637 raft_consensus.cc:348] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.662395 27637 raft_consensus.cc:374] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.662420 27637 raft_consensus.cc:729] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85bb6991fe2d46dba717aba86e5841a5, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.662469 27637 consensus_queue.cc:260] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.662562 27637 ts_tablet_manager.cc:1428] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:14.661269 27669 ts_tablet_manager.cc:1939] T bd634bb328f14e6f9f984e5414a4ea56 P df2d4f0406c8473db7d36e4d081b8b6a: Deleting consensus metadata
I20260808 07:00:14.663044 27633 tablet_bootstrap.cc:654] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.663118 26865 catalog_manager.cc:4841] TS df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397): tablet bd634bb328f14e6f9f984e5414a4ea56 (table test-table [id=73f588a979f44c9b8df2c5a05208071d]) successfully deleted
I20260808 07:00:14.664419 27633 tablet_bootstrap.cc:492] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: No bootstrap required, opened a new log
I20260808 07:00:14.664463 27633 ts_tablet_manager.cc:1397] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:00:14.664839 27633 raft_consensus.cc:348] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.664906 27633 raft_consensus.cc:374] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.664920 27633 raft_consensus.cc:729] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.664986 27633 consensus_queue.cc:260] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.665083 27633 ts_tablet_manager.cc:1428] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260808 07:00:14.698427 27193 tablet.cc:2369] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:14.770533 27643 raft_consensus.cc:482] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:14.770625 27643 raft_consensus.cc:504] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.770776 27643 leader_election.cc:288] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.771062 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fc54963e334456acb95c313fe6f3c4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5" is_pre_election: true
I20260808 07:00:14.771175 27332 raft_consensus.cc:2454] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.771085 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fc54963e334456acb95c313fe6f3c4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" is_pre_election: true
I20260808 07:00:14.771261 27128 raft_consensus.cc:2454] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.771370 27434 leader_election.cc:302] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85bb6991fe2d46dba717aba86e5841a5, f406b2dfd93d489d9591bd2abf0f4292; no voters:
I20260808 07:00:14.771502 27643 raft_consensus.cc:2781] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:14.771585 27643 raft_consensus.cc:482] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:14.771629 27643 raft_consensus.cc:3037] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.773205 27643 raft_consensus.cc:504] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.773422 27643 leader_election.cc:288] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Requested vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.773630 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fc54963e334456acb95c313fe6f3c4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5"
I20260808 07:00:14.773715 27332 raft_consensus.cc:3037] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.773698 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fc54963e334456acb95c313fe6f3c4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
I20260808 07:00:14.773801 27128 raft_consensus.cc:3037] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:14.774614 27332 raft_consensus.cc:2454] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:14.774792 27434 leader_election.cc:302] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85bb6991fe2d46dba717aba86e5841a5, f406b2dfd93d489d9591bd2abf0f4292; no voters:
I20260808 07:00:14.774883 27643 raft_consensus.cc:2781] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:14.774874 27128 raft_consensus.cc:2454] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:14.774947 27643 raft_consensus.cc:686] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 LEADER]: Becoming Leader. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Running, Role: LEADER
I20260808 07:00:14.775003 27643 consensus_queue.cc:237] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.775800 26891 catalog_manager.cc:5495] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 reported cstate change: term changed from 0 to 1, leader changed from <none> to f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } health_report { overall_health: HEALTHY } } }
W20260808 07:00:14.784878 27600 tablet.cc:2369] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:00:14 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:14.823045 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.815759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.24.181.126:36053)
0808 07:00:14.815847 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:14.815849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:14.816120 (+ 271us) server_negotiation.cc:407] Connection header received
0808 07:00:14.816192 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:14.816192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:14.816203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:14.816279 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:14.816846 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.817344 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.818039 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.818225 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.818312 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.820576 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:14.820577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:14.820578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:14.820585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:14.821799 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.822184 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.822185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.822186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.822246 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:14.822442 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:14.822443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:14.822444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:14.822554 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:14.822632 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:14.822791 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:00:14.822918 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:00:14.823143 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:14.815755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36053 (local address 127.0.0.1:36018)
0808 07:00:14.816023 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:14.816034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:14.816134 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:14.816283 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:14.816285 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:14.816676 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:14.816731 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.816734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.817427 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.817428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.817948 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:14.817949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:14.818312 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:14.818313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:14.818365 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:14.818987 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:14.818992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:14.820481 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:14.821894 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.821902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.821904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.822097 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.822313 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:14.822314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:14.822315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:14.822376 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:14.822623 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:14.822631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:14.822677 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:00:14.822768 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:00:14.823013 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":129,"thread_start_us":142,"threads_started":1}
I20260808 07:00:14.825273 26891 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:36018:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:00:14.825601 26891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:14.827145 27101 tablet_service.cc:1467] Processing CreateTablet for tablet d9273585ddaa409e90eb46a37e2ffcf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71522e7d6bad43c1b9a8356125830f6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:14.827176 27303 tablet_service.cc:1467] Processing CreateTablet for tablet d9273585ddaa409e90eb46a37e2ffcf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71522e7d6bad43c1b9a8356125830f6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:14.827343 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9273585ddaa409e90eb46a37e2ffcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.827344 27303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9273585ddaa409e90eb46a37e2ffcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.829658 27637 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: Bootstrap starting.
I20260808 07:00:14.829658 27635 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: Bootstrap starting.
I20260808 07:00:14.829811 27504 tablet_service.cc:1467] Processing CreateTablet for tablet d9273585ddaa409e90eb46a37e2ffcf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71522e7d6bad43c1b9a8356125830f6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:14.829949 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9273585ddaa409e90eb46a37e2ffcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:14.830977 27635 tablet_bootstrap.cc:654] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.831264 27637 tablet_bootstrap.cc:654] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.831626 27633 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: Bootstrap starting.
I20260808 07:00:14.832140 27633 tablet_bootstrap.cc:654] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:14.832888 27637 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: No bootstrap required, opened a new log
I20260808 07:00:14.832966 27637 ts_tablet_manager.cc:1397] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:00:14.833052 27635 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: No bootstrap required, opened a new log
I20260808 07:00:14.833132 27635 ts_tablet_manager.cc:1397] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 07:00:14.833415 27633 tablet_bootstrap.cc:492] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: No bootstrap required, opened a new log
I20260808 07:00:14.833417 27637 raft_consensus.cc:348] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833495 27633 ts_tablet_manager.cc:1397] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:14.833506 27637 raft_consensus.cc:374] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.833532 27637 raft_consensus.cc:729] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85bb6991fe2d46dba717aba86e5841a5, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.833534 27635 raft_consensus.cc:348] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833585 27637 consensus_queue.cc:260] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833619 27635 raft_consensus.cc:374] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.833643 27635 raft_consensus.cc:729] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2d4f0406c8473db7d36e4d081b8b6a, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.833604 27633 raft_consensus.cc:348] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833670 27633 raft_consensus.cc:374] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:14.833691 27633 raft_consensus.cc:729] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Initialized, Role: FOLLOWER
I20260808 07:00:14.833698 27637 ts_tablet_manager.cc:1428] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:14.833689 27635 consensus_queue.cc:260] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833735 27633 consensus_queue.cc:260] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.833801 27635 ts_tablet_manager.cc:1428] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:14.833828 27633 ts_tablet_manager.cc:1428] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:14.833971 27648 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:14.834039 27644 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:14.834093 27648 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:14.833982 27645 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:14.834124 27644 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:14.834141 27645 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:14.995299 27648 raft_consensus.cc:482] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:14.995422 27648 raft_consensus.cc:504] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:14.995609 27648 leader_election.cc:288] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:14.995927 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9273585ddaa409e90eb46a37e2ffcf4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" is_pre_election: true
I20260808 07:00:14.995945 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9273585ddaa409e90eb46a37e2ffcf4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5" is_pre_election: true
I20260808 07:00:14.996075 27128 raft_consensus.cc:2454] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.996083 27332 raft_consensus.cc:2454] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 0.
I20260808 07:00:14.996285 27434 leader_election.cc:302] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df2d4f0406c8473db7d36e4d081b8b6a, f406b2dfd93d489d9591bd2abf0f4292; no voters:
I20260808 07:00:14.996424 27648 raft_consensus.cc:2781] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:14.996479 27648 raft_consensus.cc:482] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:14.996512 27648 raft_consensus.cc:3037] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:15.041718 27648 raft_consensus.cc:504] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:15.041992 27648 leader_election.cc:288] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Requested vote from peers df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397), 85bb6991fe2d46dba717aba86e5841a5 (127.24.181.66:45329)
I20260808 07:00:15.042243 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9273585ddaa409e90eb46a37e2ffcf4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d4f0406c8473db7d36e4d081b8b6a"
I20260808 07:00:15.042282 27332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9273585ddaa409e90eb46a37e2ffcf4" candidate_uuid: "f406b2dfd93d489d9591bd2abf0f4292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85bb6991fe2d46dba717aba86e5841a5"
I20260808 07:00:15.042367 27332 raft_consensus.cc:3037] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:15.042367 27128 raft_consensus.cc:3037] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:15.043284 27332 raft_consensus.cc:2454] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:15.043385 27128 raft_consensus.cc:2454] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f406b2dfd93d489d9591bd2abf0f4292 in term 1.
I20260808 07:00:15.043533 27434 leader_election.cc:302] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85bb6991fe2d46dba717aba86e5841a5, f406b2dfd93d489d9591bd2abf0f4292; no voters:
I20260808 07:00:15.043690 27648 raft_consensus.cc:2781] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:15.043767 27648 raft_consensus.cc:686] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 LEADER]: Becoming Leader. State: Replica: f406b2dfd93d489d9591bd2abf0f4292, State: Running, Role: LEADER
I20260808 07:00:15.043830 27648 consensus_queue.cc:237] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } }
I20260808 07:00:15.044154 27643 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: TxnStatusTablet state changed. Reason: New leader f406b2dfd93d489d9591bd2abf0f4292. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.044239 27643 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:15.044399 27695 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:00:15.044555 27695 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:00:15.044584 26892 catalog_manager.cc:5495] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 reported cstate change: term changed from 0 to 1, leader changed from <none> to f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:15.060055 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.057427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.24.181.67:33895)
0808 07:00:15.057507 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:15.057509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:15.057519 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:15.057597 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:15.057598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:15.057612 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:15.057669 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:15.057815 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.058388 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.059299 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.059495 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.059591 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.059782 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:15.059826 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:15.059883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:15.059952 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:00:15.059976 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:00:15.060073 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.057370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:33895 (local address 127.0.0.1:55698)
0808 07:00:15.057463 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:15.057477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:15.057529 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:15.057683 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:15.057684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:15.057685 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:15.057737 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:15.057740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.058477 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.058479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:15.059199 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:15.059202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.059594 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.059595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:15.059669 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.059685 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:15.059898 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:15.059908 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:00:15.059951 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:00:15.060000 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:00:15.061295 27332 raft_consensus.cc:1261] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:15.061322 27128 raft_consensus.cc:1261] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:15.061537 27648 consensus_queue.cc:1035] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:15.061664 27648 consensus_queue.cc:1035] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:15.062314 27645 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: TxnStatusTablet state changed. Reason: New leader f406b2dfd93d489d9591bd2abf0f4292. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.062415 27645 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:15.062481 27644 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: TxnStatusTablet state changed. Reason: New leader f406b2dfd93d489d9591bd2abf0f4292. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.062561 27644 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:15.063177 27662 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.063242 27644 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.063287 27662 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:15.063311 27644 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P df2d4f0406c8473db7d36e4d081b8b6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:15.064640 27662 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.064724 27662 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P f406b2dfd93d489d9591bd2abf0f4292: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:15.065085 27659 mvcc.cc:205] Tried to move back new op lower bound from 7316162212089286656 to 7316162212020105216. Current Snapshot: MvccSnapshot[applied={T|T < 7316162212089286656}]
I20260808 07:00:15.065198 27645 tablet_replica.cc:439] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f406b2dfd93d489d9591bd2abf0f4292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 } } peers { permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 } } peers { permanent_uuid: "f406b2dfd93d489d9591bd2abf0f4292" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 33895 } } }
I20260808 07:00:15.065277 27645 tablet_replica.cc:442] T d9273585ddaa409e90eb46a37e2ffcf4 P 85bb6991fe2d46dba717aba86e5841a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:15.070102 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.067745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:38059 (local address 127.24.181.67:33895)
0808 07:00:15.067784 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:15.067786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:15.067933 (+ 147us) server_negotiation.cc:407] Connection header received
0808 07:00:15.068004 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:15.068005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:15.068014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:15.068068 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:15.068173 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.068610 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.069419 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.069599 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.069677 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.069837 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:15.069866 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:15.069930 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:15.069982 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 07:00:15.070021 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":18}
I20260808 07:00:15.070119 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.067679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:33895 (local address 127.24.181.67:38059)
0808 07:00:15.067853 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:15.067869 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:15.067942 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:15.068072 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:15.068072 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:15.068074 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:15.068120 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:15.068121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.068699 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.068700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:15.069328 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:15.069330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.069679 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.069679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:15.069729 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.069750 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:15.069930 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:15.069932 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:15.069981 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:15.070034 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260808 07:00:15.072346 27332 raft_consensus.cc:1261] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:15.072389 27128 raft_consensus.cc:1261] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Refusing update from remote peer f406b2dfd93d489d9591bd2abf0f4292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:15.072530 27643 consensus_queue.cc:1035] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85bb6991fe2d46dba717aba86e5841a5" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:15.072623 27648 consensus_queue.cc:1035] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2d4f0406c8473db7d36e4d081b8b6a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:15.086869 26890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:15.087010 26890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:15.087770 26890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b5c8aa240b14ad59bf963d738aae062: Sending DeleteTablet for 3 replicas of tablet 44fc54963e334456acb95c313fe6f3c4
I20260808 07:00:15.088050 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fc54963e334456acb95c313fe6f3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:15 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55668
I20260808 07:00:15.088095 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fc54963e334456acb95c313fe6f3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:15 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:35734
I20260808 07:00:15.088155 27668 tablet_replica.cc:330] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: stopping tablet replica
I20260808 07:00:15.088167 27303 tablet_service.cc:1514] Processing DeleteTablet for tablet 44fc54963e334456acb95c313fe6f3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:15 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:60106
I20260808 07:00:15.088193 27669 tablet_replica.cc:330] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: stopping tablet replica
I20260808 07:00:15.088227 27668 raft_consensus.cc:2229] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:15.088238 27669 raft_consensus.cc:2229] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:15.088274 27669 raft_consensus.cc:2258] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:15.088292 27668 raft_consensus.cc:2258] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:15.088374 27672 tablet_replica.cc:330] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: stopping tablet replica
I20260808 07:00:15.088430 27672 raft_consensus.cc:2229] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:15.088476 27672 raft_consensus.cc:2258] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:15.088640 27668 ts_tablet_manager.cc:1905] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:15.088821 27672 ts_tablet_manager.cc:1905] T 44fc54963e334456acb95c313fe6f3c4 P 85bb6991fe2d46dba717aba86e5841a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:15.088821 27669 ts_tablet_manager.cc:1905] T 44fc54963e334456acb95c313fe6f3c4 P df2d4f0406c8473db7d36e4d081b8b6a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 07:00:15 dist-test-slave-nfb5 krb5kdc[26821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:15.101768 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.091234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.24.181.65:46397)
0808 07:00:15.091281 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:15.091283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:15.091535 (+ 252us) server_negotiation.cc:407] Connection header received
0808 07:00:15.091594 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:15.091595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:15.091605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:15.091655 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:15.092408 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.092888 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.093925 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.094109 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.094162 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.097642 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:15.097652 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:15.097653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:15.097660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:15.099900 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:15.100658 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:15.100662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:15.100662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:15.100869 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:15.101104 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:15.101105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:15.101105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:15.101308 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:15.101463 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:15.101576 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:00:15.101661 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 07:00:15.102084 27181 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35750
I20260808 07:00:15.109586 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.103242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.24.181.126:36053)
0808 07:00:15.103295 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:15.103297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:15.103310 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:00:15.103389 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:15.103390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:15.103399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:15.103467 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:15.104085 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.104592 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.105565 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:15.105747 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:15.105799 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:15.107104 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:15.107106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:15.107106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:15.107112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:15.108488 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:15.108895 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:15.108896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:15.108896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:15.108949 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:15.109151 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:15.109152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:15.109152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:15.109252 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:15.109350 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:15.109460 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:00:15.109521 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 07:00:15.109938 26890 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36032
I20260808 07:00:15.110639 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 27006
I20260808 07:00:15.150195 27668 ts_tablet_manager.cc:1918] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:15.150287 27668 log.cc:1196] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/44fc54963e334456acb95c313fe6f3c4
I20260808 07:00:15.150520 27668 ts_tablet_manager.cc:1939] T 44fc54963e334456acb95c313fe6f3c4 P f406b2dfd93d489d9591bd2abf0f4292: Deleting consensus metadata
W20260808 07:00:15.152443 26865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:46397 (error 108)
I20260808 07:00:15.153115 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 27210
I20260808 07:00:15.162446 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.162198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:46397 (local address 127.0.0.1:35762)
0808 07:00:15.162322 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:15.162386 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:46397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
W20260808 07:00:15.162658 26865 catalog_manager.cc:4584] TS df2d4f0406c8473db7d36e4d081b8b6a (127.24.181.65:46397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44fc54963e334456acb95c313fe6f3c4: Network error: Client connection negotiation failed: client connection to 127.24.181.65:46397: connect: Connection refused (error 111)
I20260808 07:00:15.164860 26863 catalog_manager.cc:4841] TS f406b2dfd93d489d9591bd2abf0f4292 (127.24.181.67:33895): tablet 44fc54963e334456acb95c313fe6f3c4 (table test-table [id=11074550f21c40dfab435b64d1703299]) successfully deleted
I20260808 07:00:15.168645 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 27411
I20260808 07:00:15.168771 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:15.168602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45329 (local address 127.0.0.1:60112)
0808 07:00:15.168685 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:15.168729 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.66:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 07:00:15.176954 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 26838
2026-08-08T07:00:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2675 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:16 dist-test-slave-nfb5 krb5kdc[27724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:16 dist-test-slave-nfb5 krb5kdc[27724](info): set up 2 sockets
Aug 08 07:00:16 dist-test-slave-nfb5 krb5kdc[27724](info): commencing operation
krb5kdc: starting...
W20260808 07:00:17.892797 25301 mini_kdc.cc:121] Time spent starting KDC: real 2.690s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:17 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:18.013401 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46211
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:46211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:18.096592 27751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:18.096833 27751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:18.096872 27751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:18.098002 27751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:18.098073 27751 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:00:18.098098 27751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:18.098130 27751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:18.098179 27751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:18.098217 27751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:18.099819 27751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:46211
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:18.100098 27751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:18.100351 27751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:00:18.105439 27751 server_base.cc:1034] running on GCE node
W20260808 07:00:18.105383 27760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.105376 27758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.105383 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:18.105943 27751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:18.106235 27751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:18.107405 27751 hybrid_clock.cc:648] HybridClock initialized: now 1786172418107392 us; error 31 us; skew 500 ppm
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:18.109954 27751 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:18.110387 27751 webserver.cc:460] Webserver started at http://127.24.181.126:35277/ using document root <none> and password file <none>
I20260808 07:00:18.110638 27751 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:18.110708 27751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:18.110852 27751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:18.111948 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "6d6f144bae154b2c973c857df9dcf8ba"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "c8173230ba8af94184d3d06f5958faaa"
server_key_iv: "72622c3d63bc7795d3d8748662227ccf"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.112301 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "6d6f144bae154b2c973c857df9dcf8ba"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "c8173230ba8af94184d3d06f5958faaa"
server_key_iv: "72622c3d63bc7795d3d8748662227ccf"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.113624 27751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:18.114348 27769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.114524 27751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:18.114624 27751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "6d6f144bae154b2c973c857df9dcf8ba"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "c8173230ba8af94184d3d06f5958faaa"
server_key_iv: "72622c3d63bc7795d3d8748662227ccf"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.114704 27751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:18.136719 27751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:18.139245 27751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:18.139416 27751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:18.144099 27751 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:46211
I20260808 07:00:18.144210 27864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:46211 every 8 connection(s)
I20260808 07:00:18.144522 27751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:18.145232 27867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.147607 27867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Bootstrap starting.
I20260808 07:00:18.148222 27867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.148520 27867 log.cc:824] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:18.148902 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 27751
I20260808 07:00:18.149013 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:18.149194 25301 external_mini_cluster.cc:1428] Setting key e23d181a90a0d36baef9fa457372d080
I20260808 07:00:18.149384 27867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: No bootstrap required, opened a new log
I20260808 07:00:18.152232 27867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } }
I20260808 07:00:18.152395 27867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.152443 27867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6f144bae154b2c973c857df9dcf8ba, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.152570 27867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } }
I20260808 07:00:18.152667 27867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:18.152714 27867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:18.152760 27867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:18.153493 27867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } }
I20260808 07:00:18.153609 27867 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6f144bae154b2c973c857df9dcf8ba; no voters:
I20260808 07:00:18.153759 27867 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:18.153800 27875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:18.153978 27875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [term 1 LEADER]: Becoming Leader. State: Replica: 6d6f144bae154b2c973c857df9dcf8ba, State: Running, Role: LEADER
I20260808 07:00:18.154014 27867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:18.154121 27875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } }
I20260808 07:00:18.154559 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6f144bae154b2c973c857df9dcf8ba. Latest consensus state: current_term: 1 leader_uuid: "6d6f144bae154b2c973c857df9dcf8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } } }
I20260808 07:00:18.154728 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:18.154959 27877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d6f144bae154b2c973c857df9dcf8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6f144bae154b2c973c857df9dcf8ba" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46211 } } }
I20260808 07:00:18.155097 27877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:18.155287 27888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:18.155814 27888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.157569 27888 catalog_manager.cc:1347] Generated new cluster ID: dab75b57fa7d4b499de52869a98e3de5
I20260808 07:00:18.157629 27888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:18.161209 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.149740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.24.181.126:46211)
0808 07:00:18.149920 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.149929 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.149964 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:00:18.150733 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.150741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.150789 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.150905 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.151825 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.152844 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.153938 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.154560 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.154687 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.157875 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.157884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.157884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.157895 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.159635 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.160020 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.160023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.160023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.160112 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.160336 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.160337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.160337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.160564 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.160757 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.160867 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.161011 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260808 07:00:18.163556 27888 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:18.164239 27888 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:18.170614 27888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Generated new TSK 0
I20260808 07:00:18.170789 27888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:18.201314 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46211
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:18.306944 27905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:18.307168 27905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:18.307194 27905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:18.308389 27905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:18.308435 27905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:18.308456 27905 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:00:18.308468 27905 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:18.308486 27905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:18.310065 27905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:18.310313 27905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:18.310559 27905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:18.316329 27914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.316349 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:18.316520 27905 server_base.cc:1034] running on GCE node
W20260808 07:00:18.316411 27916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:18.316793 27905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:18.317070 27905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:18.318214 27905 hybrid_clock.cc:648] HybridClock initialized: now 1786172418318195 us; error 36 us; skew 500 ppm
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:18.320827 27905 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:18.321272 27905 webserver.cc:460] Webserver started at http://127.24.181.65:42315/ using document root <none> and password file <none>
I20260808 07:00:18.321456 27905 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:18.321497 27905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:18.321587 27905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:18.322598 27905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "0bf8b3663a4a4501a409879ab1e603ce"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "addb07283004026e4028873f6dd176c8"
server_key_iv: "8fc175df9b8f770ede7e3f41c8e4dd33"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.322970 27905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "0bf8b3663a4a4501a409879ab1e603ce"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "addb07283004026e4028873f6dd176c8"
server_key_iv: "8fc175df9b8f770ede7e3f41c8e4dd33"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.324275 27905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:18.325043 27927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.325397 27905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:18.325507 27905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "0bf8b3663a4a4501a409879ab1e603ce"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "addb07283004026e4028873f6dd176c8"
server_key_iv: "8fc175df9b8f770ede7e3f41c8e4dd33"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.325584 27905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:18.332544 27905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:18.334988 27905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:18.335160 27905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:18.335726 27905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:18.335785 27905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.335834 27905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:18.335860 27905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.343344 27905 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:40759
I20260808 07:00:18.343693 27905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:18.343761 28114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:40759 every 8 connection(s)
I20260808 07:00:18.345960 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.336130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:47083 (local address 127.24.181.126:46211)
0808 07:00:18.336218 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.336221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.336455 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:00:18.336959 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.336961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.336976 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.337064 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.337743 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.338525 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.339723 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.339935 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.340016 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.343072 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.343074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.343074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.343085 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.344797 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.345165 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.345166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.345166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.345218 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.345445 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.345445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.345445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.345548 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.345598 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.345826 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.345895 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:00:18.346133 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.336131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.65:47083)
0808 07:00:18.336311 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.336351 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.336826 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.337117 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.337120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.337417 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.337604 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.337608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.338624 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.338625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.339605 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.339608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.340025 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.340026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.340192 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.340731 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.340738 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.342963 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.344875 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.344880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.344882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.345096 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.345277 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.345277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.345278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.345388 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.345602 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.345606 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.345719 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.345800 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.345921 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
I20260808 07:00:18.346634 28115 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46211
I20260808 07:00:18.346719 28115 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:18.346892 28115 heartbeater.cc:511] Master 127.24.181.126:46211 requested a full tablet report, sending...
I20260808 07:00:18.347141 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 27905
I20260808 07:00:18.347276 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:18.347482 25301 external_mini_cluster.cc:1428] Setting key 87f12d021a2e28446a02ad1547fb5ce2
I20260808 07:00:18.347509 27797 ts_manager.cc:194] Registered new tserver with Master: 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759)
I20260808 07:00:18.348290 27797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:47083
I20260808 07:00:18.355010 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.347815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.65:48797)
0808 07:00:18.347942 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.347952 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.348002 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.348169 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.348170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.348385 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.348473 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.348475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.349406 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.349407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.350197 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.350199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.350575 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.350576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.350652 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.351184 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.351190 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.351843 (+ 653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.353825 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.353826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.353827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.353998 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.354226 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.354226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.354227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.354279 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.354628 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.354629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.354693 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.354750 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.354882 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260808 07:00:18.355020 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.347998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:48797 (local address 127.24.181.126:46211)
0808 07:00:18.348058 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.348059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.348066 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:00:18.348084 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.348085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.348094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.348165 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.348574 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.349283 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.350294 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.350500 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.350578 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.351969 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.351970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.351995 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.352003 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.353738 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.354076 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.354077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.354078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.354144 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.354346 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.354347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.354348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.354499 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.354579 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.354811 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.354889 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:18.374439 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46211
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:18.457513 28129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:18.457759 28129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:18.457805 28129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:18.459012 28129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:18.459079 28129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:18.459112 28129 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:00:18.459144 28129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:18.459177 28129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:18.460636 28129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:18.460953 28129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:18.461184 28129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:18.466228 28138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.466235 28137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.466218 28140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:18.466390 28129 server_base.cc:1034] running on GCE node
I20260808 07:00:18.466681 28129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:18.466974 28129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:18.468189 28129 hybrid_clock.cc:648] HybridClock initialized: now 1786172418468173 us; error 26 us; skew 500 ppm
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:18.470698 28129 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:18.471237 28129 webserver.cc:460] Webserver started at http://127.24.181.66:40169/ using document root <none> and password file <none>
I20260808 07:00:18.471486 28129 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:18.471558 28129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:18.471722 28129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:18.472766 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "18d51b20a2c444f2a6f4750e5944a50b"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "9389a438e529cf4008e7b8b1e004926a"
server_key_iv: "9d64f5297c3aad8ba3d613b402ab76fe"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.473131 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "18d51b20a2c444f2a6f4750e5944a50b"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "9389a438e529cf4008e7b8b1e004926a"
server_key_iv: "9d64f5297c3aad8ba3d613b402ab76fe"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.474463 28129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:00:18.475271 28148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.475437 28129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.475539 28129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "18d51b20a2c444f2a6f4750e5944a50b"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "9389a438e529cf4008e7b8b1e004926a"
server_key_iv: "9d64f5297c3aad8ba3d613b402ab76fe"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.475618 28129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:18.483070 28129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:18.484876 28129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:18.485040 28129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:18.485611 28129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:18.485671 28129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.485716 28129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:18.485746 28129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.493495 28129 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:36027
I20260808 07:00:18.493585 28314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:36027 every 8 connection(s)
I20260808 07:00:18.493955 28129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:18.496389 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.485972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:57529 (local address 127.24.181.126:46211)
0808 07:00:18.486028 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.486030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.486296 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:00:18.487065 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.487066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.487077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.487117 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.488058 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.488565 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.490232 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.490445 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.490541 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.493299 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.493301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.493302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.493309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.494879 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.495341 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.495342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.495343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.495408 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.495696 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.495697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.495697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.495840 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.495922 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.496205 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.496288 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:18.496559 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.485961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.66:57529)
0808 07:00:18.486146 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.486191 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.486958 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.487201 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.487206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.487617 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.487947 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.487953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.488648 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.488649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.490130 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.490133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.490565 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.490567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.490702 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.491301 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.491310 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.493196 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.495016 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.495023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.495025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.495254 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.495483 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.495484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.495485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.495596 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.495927 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.495935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.496070 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.496173 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.496330 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260808 07:00:18.497063 28315 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46211
I20260808 07:00:18.497182 28315 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:18.497354 28315 heartbeater.cc:511] Master 127.24.181.126:46211 requested a full tablet report, sending...
I20260808 07:00:18.497789 27798 ts_manager.cc:194] Registered new tserver with Master: 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027)
I20260808 07:00:18.498238 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:57529
I20260808 07:00:18.499348 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 28129
I20260808 07:00:18.499478 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:18.499722 25301 external_mini_cluster.cc:1428] Setting key b9a38e12cf03e56a22cd929bca2eb840
I20260808 07:00:18.504194 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.498069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.66:59931)
0808 07:00:18.498261 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.498270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.498418 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.498544 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.498545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.498726 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.498795 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.498797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.499493 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.499494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.500109 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.500111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.500499 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.500500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.500587 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.500988 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.500994 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.501404 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.503096 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.503097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.503098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.503340 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.503560 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.503560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.503561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.503618 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.503962 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.503963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.503998 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.504049 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.504126 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260808 07:00:18.504298 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.498162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:59931 (local address 127.24.181.126:46211)
0808 07:00:18.498200 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.498202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.498334 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:00:18.498482 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.498483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.498493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.498541 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.498874 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.499397 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.500198 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.500442 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.500491 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.501489 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.501490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.501491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.501497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.503018 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.503417 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.503418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.503418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.503477 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.503689 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.503690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.503690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.503892 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.503969 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.504078 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.504161 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:18.527663 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46211
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:18.610162 28329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:18.610407 28329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:18.610450 28329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:18.611620 28329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:18.611687 28329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:18.611720 28329 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:00:18.611749 28329 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:18.611783 28329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:18.613258 28329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:18.613518 28329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:18.613761 28329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:18.618860 28334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.618870 28335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:18.618872 28337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:18.619177 28329 server_base.cc:1034] running on GCE node
I20260808 07:00:18.619364 28329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:18.619549 28329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:18.620688 28329 hybrid_clock.cc:648] HybridClock initialized: now 1786172418620668 us; error 36 us; skew 500 ppm
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:18.623178 28329 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:18.623678 28329 webserver.cc:460] Webserver started at http://127.24.181.67:40185/ using document root <none> and password file <none>
I20260808 07:00:18.623912 28329 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:18.623957 28329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:18.624102 28329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:18.625195 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "500678191d4f4a168ceea97086bb1ad0"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "a1afb103b1e7907d48829d48702c9a1a"
server_key_iv: "851b7e99d66d8080aa8d6396d1eb38a4"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.625569 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "500678191d4f4a168ceea97086bb1ad0"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "a1afb103b1e7907d48829d48702c9a1a"
server_key_iv: "851b7e99d66d8080aa8d6396d1eb38a4"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.626895 28329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:18.627764 28345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.627949 28329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:18.628057 28329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "500678191d4f4a168ceea97086bb1ad0"
format_stamp: "Formatted at 2026-08-08 07:00:18 on dist-test-slave-nfb5"
server_key: "a1afb103b1e7907d48829d48702c9a1a"
server_key_iv: "851b7e99d66d8080aa8d6396d1eb38a4"
server_key_version: "encryptionkey@0"
I20260808 07:00:18.628140 28329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:18.642812 28329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:18.644675 28329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:18.644840 28329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:18.645408 28329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:18.645470 28329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.645517 28329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:18.645538 28329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.653050 28329 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:37849
I20260808 07:00:18.653129 28507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:37849 every 8 connection(s)
I20260808 07:00:18.653429 28329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:18.656324 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.645835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:51677 (local address 127.24.181.126:46211)
0808 07:00:18.645905 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.645908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.646227 (+ 319us) server_negotiation.cc:407] Connection header received
0808 07:00:18.647061 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.647062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.647078 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.647147 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.648065 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.648891 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.650553 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.650882 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.651018 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.653424 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.653426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.653427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.653434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.655045 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.655519 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.655520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.655520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.655589 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.655793 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.655793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.655793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.655896 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.655949 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.656191 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.656261 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:18.656597 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.645872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.67:51677)
0808 07:00:18.646070 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.646125 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.646872 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.647227 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.647232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.647644 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.647964 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.647970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.649005 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.649006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.650447 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.650451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.651025 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.651026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.651175 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.651727 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.651735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.653327 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.655123 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.655129 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.655131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.655421 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.655636 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.655637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.655637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.655736 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.655938 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.655941 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.656064 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.656169 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.656343 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260808 07:00:18.657186 28509 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46211
I20260808 07:00:18.657274 28509 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:18.657456 28509 heartbeater.cc:511] Master 127.24.181.126:46211 requested a full tablet report, sending...
I20260808 07:00:18.657856 27798 ts_manager.cc:194] Registered new tserver with Master: 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:18.658342 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:51677
I20260808 07:00:18.662686 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 28329
I20260808 07:00:18.662779 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:18.663038 25301 external_mini_cluster.cc:1428] Setting key 8b859b299bcdba5762a8b7625a06b030
I20260808 07:00:18.663987 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:18.664166 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.658273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:35459 (local address 127.24.181.126:46211)
0808 07:00:18.658312 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.658313 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.658390 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:00:18.658461 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.658462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.658473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.658523 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.658968 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.659424 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.660136 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.660316 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.660373 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.661595 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.661596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.661596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.661602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.662861 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.663295 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.663296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.663296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.663359 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.663586 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.663586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.663587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.663717 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.663796 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.663973 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.664092 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:00:18.664201 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.658178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.24.181.67:35459)
0808 07:00:18.658323 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.658334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.658397 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.658529 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.658530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.658754 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.658858 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.658861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.659581 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.659582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.660058 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.660059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.660387 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.660387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.660548 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.661039 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.661063 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.661517 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.662982 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.662983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.662984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.663214 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.663428 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.663429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.663429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.663499 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.663801 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.663802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.663854 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.663927 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.664039 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.688175 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.681025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.24.181.126:46211)
0808 07:00:18.681101 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.681104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.681113 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:18.681144 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.681145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.681154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.681217 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.681764 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.682236 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.682989 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.683165 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.683227 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.685402 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.685404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.685404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.685412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.686986 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.687362 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.687363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.687363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.687423 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.687630 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.687631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.687632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.687744 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.687854 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.687986 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.688063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:00:18.689760 27798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:18.691454 27798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:18.698269 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.693857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:40759 (local address 127.0.0.1:48348)
0808 07:00:18.694393 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.694403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.694447 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.694663 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.694664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.694667 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:18.694711 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.694712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.696006 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.696007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.697007 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.697010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.697923 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.697924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.698044 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.698061 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.698142 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.698193 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":100,"thread_start_us":84,"threads_started":1}
I20260808 07:00:18.698362 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.694272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.24.181.65:40759)
0808 07:00:18.694509 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.694512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.694524 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:00:18.694565 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.694567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.694582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.694667 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:18.694801 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.695887 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.697123 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.697830 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.697910 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.697967 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.698212 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.698261 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260808 07:00:18.698602 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.694045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.0.0.1:40670)
0808 07:00:18.694751 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.694762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.694826 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.695081 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.695082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.695084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:18.695125 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.695126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.695985 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.695986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.697381 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.697384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.698379 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.698380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.698448 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.698454 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.698506 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.698545 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":39,"thread_start_us":68,"threads_started":1}
I20260808 07:00:18.699128 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.694179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.24.181.67:37849)
0808 07:00:18.694412 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.694417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.694883 (+ 466us) server_negotiation.cc:407] Connection header received
0808 07:00:18.694972 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.694974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.694990 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.695083 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:18.695185 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.695861 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.697472 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.698217 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.698284 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.698341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.698840 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.698887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260808 07:00:18.699503 28006 tablet_service.cc:1467] Processing CreateTablet for tablet 7907fc005f754e798d2b863ff5a859df (DEFAULT_TABLE table=test-table [id=d4e501de631f43509b6949bc74c6d56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:18.699704 28418 tablet_service.cc:1467] Processing CreateTablet for tablet 7907fc005f754e798d2b863ff5a859df (DEFAULT_TABLE table=test-table [id=d4e501de631f43509b6949bc74c6d56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:18.699962 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7907fc005f754e798d2b863ff5a859df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.700109 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7907fc005f754e798d2b863ff5a859df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.702939 28541 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Bootstrap starting.
I20260808 07:00:18.703572 28542 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Bootstrap starting.
I20260808 07:00:18.703984 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.693844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36027 (local address 127.0.0.1:55706)
0808 07:00:18.693996 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.694039 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.694114 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.694507 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.694510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.694512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:18.694578 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.694580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.695953 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.695954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.697565 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.697567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.703703 (+ 6136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.703704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.703769 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.703773 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.703834 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.703882 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":46,"thread_start_us":45,"threads_started":1}
I20260808 07:00:18.703981 28541 tablet_bootstrap.cc:654] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.704231 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.694025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.24.181.66:36027)
0808 07:00:18.694237 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.694242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.694254 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:00:18.694296 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.694298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.694315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.694418 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:18.694663 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.695846 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.697648 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.698725 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.699009 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.699080 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.703877 (+ 4797us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.703935 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260808 07:00:18.704370 28541 log.cc:824] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:18.704397 28542 tablet_bootstrap.cc:654] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.704727 28542 log.cc:824] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:18.704856 28222 tablet_service.cc:1467] Processing CreateTablet for tablet 7907fc005f754e798d2b863ff5a859df (DEFAULT_TABLE table=test-table [id=d4e501de631f43509b6949bc74c6d56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:18.705155 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7907fc005f754e798d2b863ff5a859df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.705426 28541 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: No bootstrap required, opened a new log
I20260808 07:00:18.705515 28541 ts_tablet_manager.cc:1397] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:00:18.705539 28542 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: No bootstrap required, opened a new log
I20260808 07:00:18.705608 28542 ts_tablet_manager.cc:1397] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:18.706974 28541 raft_consensus.cc:348] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.707252 28541 raft_consensus.cc:374] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.707304 28541 raft_consensus.cc:729] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.707423 28541 consensus_queue.cc:260] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.707635 28542 raft_consensus.cc:348] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.707729 28541 ts_tablet_manager.cc:1428] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:00:18.707742 28509 heartbeater.cc:503] Master 127.24.181.126:46211 was elected leader, sending a full tablet report...
I20260808 07:00:18.707818 28542 raft_consensus.cc:374] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.707872 28542 raft_consensus.cc:729] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.707995 28542 consensus_queue.cc:260] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.708307 28542 ts_tablet_manager.cc:1428] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:18.708323 28115 heartbeater.cc:503] Master 127.24.181.126:46211 was elected leader, sending a full tablet report...
I20260808 07:00:18.709263 28549 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Bootstrap starting.
I20260808 07:00:18.709785 28549 tablet_bootstrap.cc:654] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.710037 28549 log.cc:824] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:18.710681 28549 tablet_bootstrap.cc:492] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: No bootstrap required, opened a new log
I20260808 07:00:18.710736 28549 ts_tablet_manager.cc:1397] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:18.712363 28549 raft_consensus.cc:348] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.712504 28549 raft_consensus.cc:374] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.712527 28549 raft_consensus.cc:729] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.712630 28549 consensus_queue.cc:260] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.712898 28549 ts_tablet_manager.cc:1428] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:00:18.712965 28315 heartbeater.cc:503] Master 127.24.181.126:46211 was elected leader, sending a full tablet report...
W20260808 07:00:18.745319 28317 tablet.cc:2369] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:18.836336 28552 raft_consensus.cc:482] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:18.836478 28552 raft_consensus.cc:504] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.836858 28552 leader_election.cc:288] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:18.840971 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.837262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37277 (local address 127.24.181.67:37849)
0808 07:00:18.837331 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.837333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.837344 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:18.837383 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.837389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.837397 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.837475 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:18.837670 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.838443 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840011 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.840575 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840675 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.840684 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.840858 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.840898 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:00:18.840971 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.837156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:44521 (local address 127.24.181.65:40759)
0808 07:00:18.837205 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.837207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.837470 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:00:18.837521 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.837522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.837530 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.837586 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:18.837741 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.838446 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840009 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.840585 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840675 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.840684 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.840858 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.840898 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:18.840984 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.837083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.24.181.66:37277)
0808 07:00:18.837150 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.837165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.837221 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.837483 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.837484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.837485 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:18.837593 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.837596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.838534 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.838535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.839910 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.839914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.840681 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.840776 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.840778 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.840833 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.840884 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":52}
I20260808 07:00:18.840993 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.837089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:40759 (local address 127.24.181.66:44521)
0808 07:00:18.837390 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.837398 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.837470 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.837596 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.837597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.837598 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:18.837667 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.837668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.838534 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.838535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.839910 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.839914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.840682 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.840682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.840767 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.840771 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.840841 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.840884 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":182,"thread_start_us":157,"threads_started":1}
I20260808 07:00:18.841372 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7907fc005f754e798d2b863ff5a859df" candidate_uuid: "18d51b20a2c444f2a6f4750e5944a50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce" is_pre_election: true
I20260808 07:00:18.841388 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7907fc005f754e798d2b863ff5a859df" candidate_uuid: "18d51b20a2c444f2a6f4750e5944a50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0" is_pre_election: true
I20260808 07:00:18.841544 28038 raft_consensus.cc:2454] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18d51b20a2c444f2a6f4750e5944a50b in term 0.
I20260808 07:00:18.841544 28444 raft_consensus.cc:2454] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18d51b20a2c444f2a6f4750e5944a50b in term 0.
I20260808 07:00:18.841830 28152 leader_election.cc:302] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18d51b20a2c444f2a6f4750e5944a50b, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:18.842018 28552 raft_consensus.cc:2781] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:18.842099 28552 raft_consensus.cc:482] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:18.842137 28552 raft_consensus.cc:3037] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:18.843187 28552 raft_consensus.cc:504] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.843353 28552 leader_election.cc:288] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [CANDIDATE]: Term 1 election: Requested vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:18.843508 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7907fc005f754e798d2b863ff5a859df" candidate_uuid: "18d51b20a2c444f2a6f4750e5944a50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0"
I20260808 07:00:18.843539 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7907fc005f754e798d2b863ff5a859df" candidate_uuid: "18d51b20a2c444f2a6f4750e5944a50b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce"
I20260808 07:00:18.843592 28444 raft_consensus.cc:3037] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:18.843631 28038 raft_consensus.cc:3037] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:18.844337 28444 raft_consensus.cc:2454] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18d51b20a2c444f2a6f4750e5944a50b in term 1.
I20260808 07:00:18.844489 28038 raft_consensus.cc:2454] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18d51b20a2c444f2a6f4750e5944a50b in term 1.
I20260808 07:00:18.844524 28152 leader_election.cc:302] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18d51b20a2c444f2a6f4750e5944a50b, 500678191d4f4a168ceea97086bb1ad0; no voters:
W20260808 07:00:18.844537 28117 tablet.cc:2369] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:18.844631 28552 raft_consensus.cc:2781] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:18.844791 28552 raft_consensus.cc:686] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 1 LEADER]: Becoming Leader. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Running, Role: LEADER
I20260808 07:00:18.844888 28552 consensus_queue.cc:237] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.845788 27798 catalog_manager.cc:5495] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b reported cstate change: term changed from 0 to 1, leader changed from <none> to 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66). New cstate: current_term: 1 leader_uuid: "18d51b20a2c444f2a6f4750e5944a50b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:00:18 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:18.902531 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.895596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46211 (local address 127.0.0.1:59570)
0808 07:00:18.895809 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:18.895819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:18.895907 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:18.896106 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:18.896107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:18.896552 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:18.896599 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.896602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.897272 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.897273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.897804 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:18.897806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.898168 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.898169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:18.898239 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.898746 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:18.898750 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:18.900148 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:18.901581 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.901589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.901591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.901790 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.901976 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:18.901977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:18.901977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:18.902035 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:18.902240 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:18.902247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:18.902289 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:00:18.902343 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:00:18.902435 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":127,"threads_started":1}
I20260808 07:00:18.902607 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:18.895655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.24.181.126:46211)
0808 07:00:18.895754 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:18.895756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:18.895894 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:00:18.895987 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:18.895989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:18.896000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:18.896102 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:18.896688 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.897185 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.897892 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:18.898081 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:18.898167 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:18.900234 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:18.900235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:18.900235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:18.900242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:18.901510 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.901865 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.901866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.901866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.901918 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:18.902088 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:18.902088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:18.902088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:18.902186 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:18.902238 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:18.902375 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:00:18.902482 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":56}
W20260808 07:00:18.904242 28510 tablet.cc:2369] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:18.904528 27798 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:59570:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:00:18.904749 27798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:18.906284 28418 tablet_service.cc:1467] Processing CreateTablet for tablet 990d7c5eb0db4570bfe55f15d4fd5458 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=819c0a4c9c4740a8a55b8c96d3297c49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:18.906265 28222 tablet_service.cc:1467] Processing CreateTablet for tablet 990d7c5eb0db4570bfe55f15d4fd5458 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=819c0a4c9c4740a8a55b8c96d3297c49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:18.906452 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990d7c5eb0db4570bfe55f15d4fd5458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.906458 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990d7c5eb0db4570bfe55f15d4fd5458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.907090 28006 tablet_service.cc:1467] Processing CreateTablet for tablet 990d7c5eb0db4570bfe55f15d4fd5458 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=819c0a4c9c4740a8a55b8c96d3297c49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:18.907258 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990d7c5eb0db4570bfe55f15d4fd5458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:18.907868 28549 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: Bootstrap starting.
I20260808 07:00:18.908012 28541 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: Bootstrap starting.
I20260808 07:00:18.908349 28549 tablet_bootstrap.cc:654] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.908649 28541 tablet_bootstrap.cc:654] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.908910 28542 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: Bootstrap starting.
I20260808 07:00:18.909346 28542 tablet_bootstrap.cc:654] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:18.909364 28541 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: No bootstrap required, opened a new log
I20260808 07:00:18.909410 28541 ts_tablet_manager.cc:1397] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:18.909580 28541 raft_consensus.cc:348] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.909678 28541 raft_consensus.cc:374] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.909703 28541 raft_consensus.cc:729] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.909749 28541 consensus_queue.cc:260] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.909857 28541 ts_tablet_manager.cc:1428] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:18.909920 28547 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:18.909998 28547 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:18.910024 28542 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: No bootstrap required, opened a new log
I20260808 07:00:18.910072 28542 ts_tablet_manager.cc:1397] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:18.910080 28549 tablet_bootstrap.cc:492] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: No bootstrap required, opened a new log
I20260808 07:00:18.910125 28549 ts_tablet_manager.cc:1397] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:18.910262 28542 raft_consensus.cc:348] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.910281 28549 raft_consensus.cc:348] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.910343 28542 raft_consensus.cc:374] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.910363 28549 raft_consensus.cc:374] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:18.910368 28542 raft_consensus.cc:729] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.910387 28549 raft_consensus.cc:729] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Initialized, Role: FOLLOWER
I20260808 07:00:18.910414 28542 consensus_queue.cc:260] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.910429 28549 consensus_queue.cc:260] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:18.910527 28542 ts_tablet_manager.cc:1428] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:18.910528 28549 ts_tablet_manager.cc:1428] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:18.910569 28552 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:18.910633 28552 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:18.910670 28548 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:18.910754 28548 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:19.126523 28547 raft_consensus.cc:482] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:19.126638 28547 raft_consensus.cc:504] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.127142 28547 leader_election.cc:288] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027)
I20260808 07:00:19.130330 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.127354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36027 (local address 127.24.181.67:60569)
0808 07:00:19.127422 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.127437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.127510 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.127801 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.127802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.127804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:19.127882 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.127883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.128661 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.128662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.129506 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.129509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.130030 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.130031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.130131 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.130134 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.130178 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.130230 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":28}
I20260808 07:00:19.130333 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.127451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:60569 (local address 127.24.181.66:36027)
0808 07:00:19.127499 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.127500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.127513 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:00:19.127601 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.127602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.127611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.127748 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:19.127959 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.128572 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.129570 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.129948 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.130005 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.130013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.130201 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.130247 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:19.130364 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.127369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:40759 (local address 127.24.181.67:41525)
0808 07:00:19.127589 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.127601 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.127692 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.127869 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.127870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.127872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:19.127952 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.127953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.128643 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.128644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.129522 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.129525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.130091 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.130092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.130153 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.130156 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.130234 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.130270 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":88,"thread_start_us":101,"threads_started":1}
I20260808 07:00:19.130391 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.127482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:41525 (local address 127.24.181.65:40759)
0808 07:00:19.127530 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.127532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.127745 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:00:19.127777 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.127778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.127788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.127864 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:19.128037 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.128539 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.129611 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.129964 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.130058 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.130068 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.130254 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.130316 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:19.130738 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990d7c5eb0db4570bfe55f15d4fd5458" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce" is_pre_election: true
I20260808 07:00:19.130832 28038 raft_consensus.cc:2454] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 0.
I20260808 07:00:19.130790 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990d7c5eb0db4570bfe55f15d4fd5458" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b" is_pre_election: true
I20260808 07:00:19.130957 28249 raft_consensus.cc:2454] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 0.
I20260808 07:00:19.131031 28346 leader_election.cc:302] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:19.131160 28547 raft_consensus.cc:2781] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:19.131213 28547 raft_consensus.cc:482] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:19.131240 28547 raft_consensus.cc:3037] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.131969 28547 raft_consensus.cc:504] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.132118 28547 leader_election.cc:288] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027)
I20260808 07:00:19.132244 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990d7c5eb0db4570bfe55f15d4fd5458" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce"
I20260808 07:00:19.132316 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990d7c5eb0db4570bfe55f15d4fd5458" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b"
I20260808 07:00:19.132357 28038 raft_consensus.cc:3037] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.132397 28249 raft_consensus.cc:3037] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.133019 28038 raft_consensus.cc:2454] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 1.
I20260808 07:00:19.133033 28249 raft_consensus.cc:2454] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 1.
I20260808 07:00:19.133184 28346 leader_election.cc:302] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:19.133342 28547 raft_consensus.cc:2781] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:19.133507 28547 raft_consensus.cc:686] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [term 1 LEADER]: Becoming Leader. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Running, Role: LEADER
I20260808 07:00:19.133610 28547 consensus_queue.cc:237] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.134119 28575 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: TxnStatusTablet state changed. Reason: New leader 500678191d4f4a168ceea97086bb1ad0. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.134215 28575 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:19.134447 28580 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:00:19.134586 28580 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:00:19.134495 27798 catalog_manager.cc:5495] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:19.193322 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.190748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.0.0.1:40678)
0808 07:00:19.190832 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.190846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.190899 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.191123 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.191124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.191125 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:19.191179 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.191181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.191800 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.191801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.192380 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.192381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.192765 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.192766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.192839 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.192863 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.193140 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.193150 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.193196 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.193246 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:00:19.193303 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.190812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.24.181.67:37849)
0808 07:00:19.190876 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.190879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.190888 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:19.191012 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.191013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.191039 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.191117 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.191244 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.191712 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.192485 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.192676 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.192766 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.192962 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.193061 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.193149 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.193199 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.193221 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:00:19.195015 28249 raft_consensus.cc:1261] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Refusing update from remote peer 500678191d4f4a168ceea97086bb1ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.195070 28038 raft_consensus.cc:1261] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Refusing update from remote peer 500678191d4f4a168ceea97086bb1ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.195348 28547 consensus_queue.cc:1035] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.195484 28575 consensus_queue.cc:1035] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.196566 28552 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: TxnStatusTablet state changed. Reason: New leader 500678191d4f4a168ceea97086bb1ad0. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.196660 28552 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:19.197075 28548 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: TxnStatusTablet state changed. Reason: New leader 500678191d4f4a168ceea97086bb1ad0. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.197167 28548 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:19.197777 28575 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.197857 28575 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:19.197918 28583 mvcc.cc:205] Tried to move back new op lower bound from 7316162229020110848 to 7316162228771803136. Current Snapshot: MvccSnapshot[applied={T|T < 7316162229020110848}]
I20260808 07:00:19.198333 28547 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.198416 28547 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 500678191d4f4a168ceea97086bb1ad0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:19.198385 28552 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.198477 28552 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 18d51b20a2c444f2a6f4750e5944a50b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:19.198887 28548 tablet_replica.cc:439] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } } }
I20260808 07:00:19.198998 28548 tablet_replica.cc:442] T 990d7c5eb0db4570bfe55f15d4fd5458 P 0bf8b3663a4a4501a409879ab1e603ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:19.199009 28586 mvcc.cc:205] Tried to move back new op lower bound from 7316162229020110848 to 7316162228771803136. Current Snapshot: MvccSnapshot[applied={T|T < 7316162229020110848}]
I20260808 07:00:19.199119 28589 mvcc.cc:205] Tried to move back new op lower bound from 7316162229020110848 to 7316162228771803136. Current Snapshot: MvccSnapshot[applied={T|T < 7316162229020110848}]
I20260808 07:00:19.203763 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.200988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.24.181.66:36027)
0808 07:00:19.201034 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.201036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.201145 (+ 109us) server_negotiation.cc:407] Connection header received
0808 07:00:19.201215 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.201215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.201229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.201337 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.201461 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.201916 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.203060 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.203237 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.203290 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.203483 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.203569 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.203619 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.203698 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.203714 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:19.208952 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.206278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:41945 (local address 127.24.181.67:37849)
0808 07:00:19.206356 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.206358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.206487 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:00:19.206547 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.206547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.206554 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.206602 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.206739 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.207251 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.208319 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.208489 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.208536 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.208771 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.208795 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.208835 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.208890 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.208905 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:00:19.209179 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.206221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.24.181.66:41945)
0808 07:00:19.206410 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.206422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.206488 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.206606 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.206607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.206608 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:19.206669 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.206671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.207337 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.207338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.208235 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.208238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.208567 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.208568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.208640 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.208657 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.208837 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.208838 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.208889 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.208930 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260808 07:00:19.211637 28038 raft_consensus.cc:1261] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Refusing update from remote peer 18d51b20a2c444f2a6f4750e5944a50b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.211723 28444 raft_consensus.cc:1261] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Refusing update from remote peer 18d51b20a2c444f2a6f4750e5944a50b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.211917 28554 consensus_queue.cc:1035] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.212051 28552 consensus_queue.cc:1035] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [LEADER]: Connected to new peer: Peer: permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.221421 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.218519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:37047 (local address 127.24.181.66:36027)
0808 07:00:19.218559 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.218561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.218793 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:00:19.218848 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.218849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.218856 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.218902 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.219068 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.219581 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.220781 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.220962 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.221014 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.221237 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.221261 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.221298 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.221357 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.221372 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:19.221503 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.218480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36027 (local address 127.24.181.67:37047)
0808 07:00:19.218664 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.218676 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.218795 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.218906 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.218907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.218909 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:19.218996 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.218998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.219668 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.219669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.220668 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.220671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.221069 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.221071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.221137 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.221154 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.221303 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.221304 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.221356 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.221402 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260808 07:00:19.234954 27796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:19.235091 27796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:19.235972 27796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Sending DeleteTablet for 3 replicas of tablet 7907fc005f754e798d2b863ff5a859df
I20260808 07:00:19.236326 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet 7907fc005f754e798d2b863ff5a859df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:19 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:48348
I20260808 07:00:19.236406 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet 7907fc005f754e798d2b863ff5a859df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:19 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55706
I20260808 07:00:19.236709 28613 tablet_replica.cc:330] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: stopping tablet replica
I20260808 07:00:19.236716 28418 tablet_service.cc:1514] Processing DeleteTablet for tablet 7907fc005f754e798d2b863ff5a859df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:19 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40670
I20260808 07:00:19.236992 28613 raft_consensus.cc:2229] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:19.237143 28613 raft_consensus.cc:2258] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:19.237174 27796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:19.237329 27796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:19.237756 28614 tablet_replica.cc:330] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: stopping tablet replica
I20260808 07:00:19.237867 28614 raft_consensus.cc:2229] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:19.237918 28614 raft_consensus.cc:2258] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:19.238667 28614 ts_tablet_manager.cc:1905] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:19.238782 28006 tablet_service.cc:1467] Processing CreateTablet for tablet b0b98316a11e47afbee0c6364567ce1e (DEFAULT_TABLE table=test-table [id=ee660116f3384933a9ba819a68b0ba02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:19.238955 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b98316a11e47afbee0c6364567ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:19.240345 28542 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Bootstrap starting.
I20260808 07:00:19.240561 28614 ts_tablet_manager.cc:1918] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:19.240626 28614 log.cc:1196] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/7907fc005f754e798d2b863ff5a859df
I20260808 07:00:19.240761 28542 tablet_bootstrap.cc:654] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:19.240935 28614 ts_tablet_manager.cc:1939] T 7907fc005f754e798d2b863ff5a859df P 500678191d4f4a168ceea97086bb1ad0: Deleting consensus metadata
I20260808 07:00:19.241760 28542 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: No bootstrap required, opened a new log
I20260808 07:00:19.241751 28222 tablet_service.cc:1467] Processing CreateTablet for tablet b0b98316a11e47afbee0c6364567ce1e (DEFAULT_TABLE table=test-table [id=ee660116f3384933a9ba819a68b0ba02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:19.241814 28542 ts_tablet_manager.cc:1397] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:19.241868 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b98316a11e47afbee0c6364567ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:19.241966 28542 raft_consensus.cc:348] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.242048 28542 raft_consensus.cc:374] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:19.242072 28542 raft_consensus.cc:729] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Initialized, Role: FOLLOWER
I20260808 07:00:19.242116 28542 consensus_queue.cc:260] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.242203 28542 ts_tablet_manager.cc:1428] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:19.243141 28613 ts_tablet_manager.cc:1905] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:19.243211 28549 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Bootstrap starting.
I20260808 07:00:19.243273 28418 tablet_service.cc:1467] Processing CreateTablet for tablet b0b98316a11e47afbee0c6364567ce1e (DEFAULT_TABLE table=test-table [id=ee660116f3384933a9ba819a68b0ba02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:19.243389 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b98316a11e47afbee0c6364567ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:19.243681 28549 tablet_bootstrap.cc:654] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:19.244566 28541 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Bootstrap starting.
I20260808 07:00:19.244738 28612 tablet_replica.cc:330] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: stopping tablet replica
I20260808 07:00:19.244730 27776 catalog_manager.cc:4841] TS 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849): tablet 7907fc005f754e798d2b863ff5a859df (table test-table [id=d4e501de631f43509b6949bc74c6d56c]) successfully deleted
I20260808 07:00:19.244930 28612 raft_consensus.cc:2229] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:19.245031 28541 tablet_bootstrap.cc:654] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:19.245057 28612 raft_consensus.cc:2258] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:19.245209 28549 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: No bootstrap required, opened a new log
I20260808 07:00:19.245280 28549 ts_tablet_manager.cc:1397] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:19.245460 28549 raft_consensus.cc:348] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.245555 28549 raft_consensus.cc:374] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:19.245586 28549 raft_consensus.cc:729] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Initialized, Role: FOLLOWER
I20260808 07:00:19.245640 28549 consensus_queue.cc:260] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.245733 28549 ts_tablet_manager.cc:1428] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:19.245965 28612 ts_tablet_manager.cc:1905] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:19.246477 28613 ts_tablet_manager.cc:1918] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:19.247933 28613 log.cc:1196] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/7907fc005f754e798d2b863ff5a859df
I20260808 07:00:19.248282 28613 ts_tablet_manager.cc:1939] T 7907fc005f754e798d2b863ff5a859df P 18d51b20a2c444f2a6f4750e5944a50b: Deleting consensus metadata
I20260808 07:00:19.248783 27776 catalog_manager.cc:4841] TS 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027): tablet 7907fc005f754e798d2b863ff5a859df (table test-table [id=d4e501de631f43509b6949bc74c6d56c]) successfully deleted
I20260808 07:00:19.249141 28612 ts_tablet_manager.cc:1918] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:19.249267 28612 log.cc:1196] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/7907fc005f754e798d2b863ff5a859df
I20260808 07:00:19.249579 28612 ts_tablet_manager.cc:1939] T 7907fc005f754e798d2b863ff5a859df P 0bf8b3663a4a4501a409879ab1e603ce: Deleting consensus metadata
I20260808 07:00:19.250041 27770 catalog_manager.cc:4841] TS 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759): tablet 7907fc005f754e798d2b863ff5a859df (table test-table [id=d4e501de631f43509b6949bc74c6d56c]) successfully deleted
I20260808 07:00:19.251369 28541 tablet_bootstrap.cc:492] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: No bootstrap required, opened a new log
I20260808 07:00:19.251430 28541 ts_tablet_manager.cc:1397] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260808 07:00:19.251569 28541 raft_consensus.cc:348] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.251665 28541 raft_consensus.cc:374] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:19.251688 28541 raft_consensus.cc:729] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Initialized, Role: FOLLOWER
I20260808 07:00:19.251744 28541 consensus_queue.cc:260] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.251837 28541 ts_tablet_manager.cc:1428] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:19.438387 28548 raft_consensus.cc:482] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:19.438517 28548 raft_consensus.cc:504] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.438881 28548 leader_election.cc:288] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:19.442169 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.439306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:42101 (local address 127.24.181.66:36027)
0808 07:00:19.439375 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.439377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.439483 (+ 106us) server_negotiation.cc:407] Connection header received
0808 07:00:19.439590 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.439591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.439608 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.439672 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:19.439850 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.440388 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441489 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.441825 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441920 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.441931 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.442059 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.442096 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:00:19.442181 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.439551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:53135 (local address 127.24.181.67:37849)
0808 07:00:19.439631 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.439632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.439641 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:19.439746 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.439748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.439757 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.439822 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:19.439984 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.440467 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441427 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.441801 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441857 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.441864 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.442080 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.442114 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:00:19.442195 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.439433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.24.181.65:53135)
0808 07:00:19.439572 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.439583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.439654 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.439839 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.439841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.439843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:19.439912 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.439913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.440573 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.440574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.441340 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.441342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.441885 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.441987 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.441991 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.442062 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.442099 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":3,"thread_start_us":39,"threads_started":1}
I20260808 07:00:19.442198 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.439213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36027 (local address 127.24.181.65:42101)
0808 07:00:19.439394 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.439406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.439479 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.439692 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.439693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.439696 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:19.439763 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.439765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.440466 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.440467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.441366 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.441369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.441933 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.441934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.441980 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.441983 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.442026 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.442089 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":44,"thread_start_us":92,"threads_started":1}
I20260808 07:00:19.442492 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b98316a11e47afbee0c6364567ce1e" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0" is_pre_election: true
I20260808 07:00:19.442592 28444 raft_consensus.cc:2454] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 0.
I20260808 07:00:19.442713 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b98316a11e47afbee0c6364567ce1e" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b" is_pre_election: true
I20260808 07:00:19.442787 28249 raft_consensus.cc:2454] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 0.
I20260808 07:00:19.442776 27932 leader_election.cc:302] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:19.442958 28548 raft_consensus.cc:2781] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:19.443006 28548 raft_consensus.cc:482] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:19.443032 28548 raft_consensus.cc:3037] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.443861 28548 raft_consensus.cc:504] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.443996 28548 leader_election.cc:288] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 election: Requested vote from peers 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:19.444137 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b98316a11e47afbee0c6364567ce1e" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b"
I20260808 07:00:19.444172 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b98316a11e47afbee0c6364567ce1e" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0"
I20260808 07:00:19.444227 28249 raft_consensus.cc:3037] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.444242 28444 raft_consensus.cc:3037] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:19.445039 28444 raft_consensus.cc:2454] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 1.
I20260808 07:00:19.445039 28249 raft_consensus.cc:2454] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 1.
I20260808 07:00:19.445226 27932 leader_election.cc:302] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:19.445338 28548 raft_consensus.cc:2781] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:19.445541 28548 raft_consensus.cc:686] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 1 LEADER]: Becoming Leader. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Running, Role: LEADER
I20260808 07:00:19.445637 28548 consensus_queue.cc:237] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:19.446420 27798 catalog_manager.cc:5495] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65). New cstate: current_term: 1 leader_uuid: "0bf8b3663a4a4501a409879ab1e603ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } health_report { overall_health: UNKNOWN } } }
I20260808 07:00:19.477020 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.474550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.24.181.65:40759)
0808 07:00:19.474609 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.474611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.474622 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:19.474715 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.474716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.474739 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.474803 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.474971 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.475509 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.476229 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.476434 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.476506 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.476713 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.476828 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.476899 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.476935 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.476955 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:00:19.482717 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.479930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:55715 (local address 127.24.181.67:37849)
0808 07:00:19.479985 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.479987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.480162 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:00:19.480220 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.480221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.480233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.480290 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.480504 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.481022 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.481874 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.482104 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.482210 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.482427 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.482468 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.482546 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.482612 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.482633 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:00:19.482759 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.479863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.24.181.65:55715)
0808 07:00:19.480065 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.480074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.480152 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.480335 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.480336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.480337 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:19.480411 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.480412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.481118 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.481119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.481772 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.481774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.482242 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.482244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.482314 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.482330 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.482552 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.482554 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.482608 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.482664 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260808 07:00:19.484964 28444 raft_consensus.cc:1261] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Refusing update from remote peer 0bf8b3663a4a4501a409879ab1e603ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.485002 28249 raft_consensus.cc:1261] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Refusing update from remote peer 0bf8b3663a4a4501a409879ab1e603ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:19.485273 28628 consensus_queue.cc:1035] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.485476 28548 consensus_queue.cc:1035] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:19.493520 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.490988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:40759 (local address 127.24.181.67:48189)
0808 07:00:19.491056 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:19.491068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:19.491120 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:19.491264 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:19.491266 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:19.491268 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:19.491312 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.491314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.492065 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.492066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.492598 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:19.492600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.493059 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.493060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:19.493123 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.493129 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.493368 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.493370 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:19.493413 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:00:19.493467 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:00:19.493713 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:19.491011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:48189 (local address 127.24.181.65:40759)
0808 07:00:19.491056 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:19.491059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:19.491105 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:00:19.491176 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:19.491176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:19.491188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:19.491244 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:19.491373 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.491903 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.492684 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:19.492987 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:19.493052 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:19.493229 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:19.493265 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:19.493348 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:19.493611 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 07:00:19.493655 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:00:19.496914 28589 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:00:21.018112 27796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:21.018258 27796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:21.019207 27796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Sending DeleteTablet for 3 replicas of tablet b0b98316a11e47afbee0c6364567ce1e
I20260808 07:00:21.019476 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b98316a11e47afbee0c6364567ce1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:48348
I20260808 07:00:21.019482 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b98316a11e47afbee0c6364567ce1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55706
I20260808 07:00:21.019526 28418 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b98316a11e47afbee0c6364567ce1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40670
I20260808 07:00:21.022356 28680 tablet_replica.cc:330] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: stopping tablet replica
I20260808 07:00:21.022429 28680 raft_consensus.cc:2229] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:21.022526 28680 raft_consensus.cc:2258] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.022639 28679 tablet_replica.cc:330] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: stopping tablet replica
I20260808 07:00:21.022702 28679 raft_consensus.cc:2229] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:21.022749 28679 raft_consensus.cc:2258] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.022987 28680 ts_tablet_manager.cc:1905] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.023047 28679 ts_tablet_manager.cc:1905] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.024686 28679 ts_tablet_manager.cc:1918] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:00:21.024765 28679 log.cc:1196] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/b0b98316a11e47afbee0c6364567ce1e
I20260808 07:00:21.024969 28680 ts_tablet_manager.cc:1918] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:00:21.025005 28679 ts_tablet_manager.cc:1939] T b0b98316a11e47afbee0c6364567ce1e P 18d51b20a2c444f2a6f4750e5944a50b: Deleting consensus metadata
I20260808 07:00:21.025041 28680 log.cc:1196] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/b0b98316a11e47afbee0c6364567ce1e
I20260808 07:00:21.025350 28680 ts_tablet_manager.cc:1939] T b0b98316a11e47afbee0c6364567ce1e P 0bf8b3663a4a4501a409879ab1e603ce: Deleting consensus metadata
I20260808 07:00:21.025365 27776 catalog_manager.cc:4841] TS 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027): tablet b0b98316a11e47afbee0c6364567ce1e (table test-table [id=ee660116f3384933a9ba819a68b0ba02]) successfully deleted
I20260808 07:00:21.025632 28681 tablet_replica.cc:330] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: stopping tablet replica
I20260808 07:00:21.025662 27770 catalog_manager.cc:4841] TS 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759): tablet b0b98316a11e47afbee0c6364567ce1e (table test-table [id=ee660116f3384933a9ba819a68b0ba02]) successfully deleted
I20260808 07:00:21.025704 28681 raft_consensus.cc:2229] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:21.025740 28681 raft_consensus.cc:2258] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.025961 28681 ts_tablet_manager.cc:1905] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.027643 28681 ts_tablet_manager.cc:1918] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:00:21.027716 28681 log.cc:1196] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/b0b98316a11e47afbee0c6364567ce1e
I20260808 07:00:21.027944 28681 ts_tablet_manager.cc:1939] T b0b98316a11e47afbee0c6364567ce1e P 500678191d4f4a168ceea97086bb1ad0: Deleting consensus metadata
I20260808 07:00:21.028370 27776 catalog_manager.cc:4841] TS 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849): tablet b0b98316a11e47afbee0c6364567ce1e (table test-table [id=ee660116f3384933a9ba819a68b0ba02]) successfully deleted
Aug 08 07:00:21 dist-test-slave-nfb5 krb5kdc[27724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:00:21 dist-test-slave-nfb5 krb5kdc[27724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:21.045150 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:21.037958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.24.181.126:46211)
0808 07:00:21.038101 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:21.038103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:21.038152 (+ 49us) server_negotiation.cc:407] Connection header received
0808 07:00:21.038195 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:21.038196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:21.038205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:21.038277 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:21.038856 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.039459 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.040191 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.040390 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.040444 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:21.042734 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:21.042735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:21.042736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:21.042742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:21.044162 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:21.044482 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:21.044483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:21.044483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:21.044544 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:21.044716 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:21.044716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:21.044717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:21.044858 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:21.044935 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:21.045021 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:00:21.045083 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260808 07:00:21.046655 27796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:21.046856 27796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:21.048465 28222 tablet_service.cc:1467] Processing CreateTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba (DEFAULT_TABLE table=test-table [id=d3d854eb320b435cbc8a037d91d27848]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.048475 28006 tablet_service.cc:1467] Processing CreateTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba (DEFAULT_TABLE table=test-table [id=d3d854eb320b435cbc8a037d91d27848]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.048506 28418 tablet_service.cc:1467] Processing CreateTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba (DEFAULT_TABLE table=test-table [id=d3d854eb320b435cbc8a037d91d27848]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.048628 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e6b9ba792c437cb5edca7b8aa267ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.048628 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e6b9ba792c437cb5edca7b8aa267ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.048630 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e6b9ba792c437cb5edca7b8aa267ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.049819 28691 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Bootstrap starting.
I20260808 07:00:21.050246 28692 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Bootstrap starting.
I20260808 07:00:21.050371 28691 tablet_bootstrap.cc:654] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.050622 28692 tablet_bootstrap.cc:654] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.050700 28693 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Bootstrap starting.
I20260808 07:00:21.051093 28693 tablet_bootstrap.cc:654] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.051086 28691 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: No bootstrap required, opened a new log
I20260808 07:00:21.051156 28691 ts_tablet_manager.cc:1397] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:21.051301 28692 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: No bootstrap required, opened a new log
I20260808 07:00:21.051343 28692 ts_tablet_manager.cc:1397] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:21.051328 28691 raft_consensus.cc:348] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.051406 28691 raft_consensus.cc:374] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.051427 28691 raft_consensus.cc:729] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.051442 28692 raft_consensus.cc:348] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.051505 28692 raft_consensus.cc:374] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.051479 28691 consensus_queue.cc:260] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.051527 28692 raft_consensus.cc:729] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.051574 28692 consensus_queue.cc:260] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.051663 28691 ts_tablet_manager.cc:1428] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:21.051692 28692 ts_tablet_manager.cc:1428] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:21.051828 28693 tablet_bootstrap.cc:492] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: No bootstrap required, opened a new log
I20260808 07:00:21.051899 28693 ts_tablet_manager.cc:1397] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:21.052016 28693 raft_consensus.cc:348] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.052219 28693 raft_consensus.cc:374] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.052309 28693 raft_consensus.cc:729] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.052377 28693 consensus_queue.cc:260] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.052490 28693 ts_tablet_manager.cc:1428] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:21.198783 28671 raft_consensus.cc:482] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:21.198889 28671 raft_consensus.cc:504] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.199134 28671 leader_election.cc:288] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:21.199405 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e6b9ba792c437cb5edca7b8aa267ba" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b" is_pre_election: true
I20260808 07:00:21.199462 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e6b9ba792c437cb5edca7b8aa267ba" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0" is_pre_election: true
I20260808 07:00:21.199535 28249 raft_consensus.cc:2454] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 0.
I20260808 07:00:21.199589 28444 raft_consensus.cc:2454] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 0.
I20260808 07:00:21.199762 27932 leader_election.cc:302] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 18d51b20a2c444f2a6f4750e5944a50b; no voters:
I20260808 07:00:21.199867 28671 raft_consensus.cc:2781] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:21.199909 28671 raft_consensus.cc:482] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:21.199954 28671 raft_consensus.cc:3037] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.200749 28671 raft_consensus.cc:504] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.200884 28671 leader_election.cc:288] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 election: Requested vote from peers 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027), 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849)
I20260808 07:00:21.201115 28444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e6b9ba792c437cb5edca7b8aa267ba" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500678191d4f4a168ceea97086bb1ad0"
I20260808 07:00:21.201113 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e6b9ba792c437cb5edca7b8aa267ba" candidate_uuid: "0bf8b3663a4a4501a409879ab1e603ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b"
I20260808 07:00:21.201220 28444 raft_consensus.cc:3037] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.201217 28249 raft_consensus.cc:3037] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.201828 28444 raft_consensus.cc:2454] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 1.
I20260808 07:00:21.201892 28249 raft_consensus.cc:2454] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8b3663a4a4501a409879ab1e603ce in term 1.
I20260808 07:00:21.202044 27932 leader_election.cc:302] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:21.202157 28671 raft_consensus.cc:2781] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:21.202252 28671 raft_consensus.cc:686] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 1 LEADER]: Becoming Leader. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Running, Role: LEADER
I20260808 07:00:21.202320 28671 consensus_queue.cc:237] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.203023 27798 catalog_manager.cc:5495] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65). New cstate: current_term: 1 leader_uuid: "0bf8b3663a4a4501a409879ab1e603ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } health_report { overall_health: UNKNOWN } } }
I20260808 07:00:21.235548 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:21.232997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.24.181.65:40759)
0808 07:00:21.233179 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:21.233181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:21.233187 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:00:21.233211 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:21.233212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:21.233221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:21.233293 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:21.233466 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.233960 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.234760 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.235047 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.235125 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:21.235283 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:21.235319 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:21.235373 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:21.235431 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:00:21.235454 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260808 07:00:21.237850 28249 raft_consensus.cc:1261] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Refusing update from remote peer 0bf8b3663a4a4501a409879ab1e603ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:21.237905 28444 raft_consensus.cc:1261] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Refusing update from remote peer 0bf8b3663a4a4501a409879ab1e603ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:21.238106 28668 consensus_queue.cc:1035] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:21.238237 28671 consensus_queue.cc:1035] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:21.247897 27798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:21.248006 27798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:21.248586 27798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Sending DeleteTablet for 3 replicas of tablet c4e6b9ba792c437cb5edca7b8aa267ba
I20260808 07:00:21.248829 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:48348
I20260808 07:00:21.248929 28680 tablet_replica.cc:330] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: stopping tablet replica
I20260808 07:00:21.248919 28418 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40670
I20260808 07:00:21.248978 28680 raft_consensus.cc:2229] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:21.249012 28681 tablet_replica.cc:330] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: stopping tablet replica
I20260808 07:00:21.248992 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e6b9ba792c437cb5edca7b8aa267ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:21 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55706
I20260808 07:00:21.249029 28680 raft_consensus.cc:2258] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.249071 28681 raft_consensus.cc:2229] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:21.249081 28679 tablet_replica.cc:330] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: stopping tablet replica
I20260808 07:00:21.249123 28681 raft_consensus.cc:2258] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.249128 28679 raft_consensus.cc:2229] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:21.249168 28679 raft_consensus.cc:2258] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:21.249444 28681 ts_tablet_manager.cc:1905] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.249545 28679 ts_tablet_manager.cc:1905] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.249589 28680 ts_tablet_manager.cc:1905] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:21.250905 28681 ts_tablet_manager.cc:1918] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:21.251047 28681 log.cc:1196] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/c4e6b9ba792c437cb5edca7b8aa267ba
I20260808 07:00:21.251101 28680 ts_tablet_manager.cc:1918] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:21.251101 28679 ts_tablet_manager.cc:1918] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:21.251156 28680 log.cc:1196] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/c4e6b9ba792c437cb5edca7b8aa267ba
I20260808 07:00:21.251156 28679 log.cc:1196] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/c4e6b9ba792c437cb5edca7b8aa267ba
I20260808 07:00:21.251242 28681 ts_tablet_manager.cc:1939] T c4e6b9ba792c437cb5edca7b8aa267ba P 500678191d4f4a168ceea97086bb1ad0: Deleting consensus metadata
I20260808 07:00:21.251389 28679 ts_tablet_manager.cc:1939] T c4e6b9ba792c437cb5edca7b8aa267ba P 18d51b20a2c444f2a6f4750e5944a50b: Deleting consensus metadata
I20260808 07:00:21.251405 28680 ts_tablet_manager.cc:1939] T c4e6b9ba792c437cb5edca7b8aa267ba P 0bf8b3663a4a4501a409879ab1e603ce: Deleting consensus metadata
I20260808 07:00:21.251550 27776 catalog_manager.cc:4841] TS 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849): tablet c4e6b9ba792c437cb5edca7b8aa267ba (table test-table [id=d3d854eb320b435cbc8a037d91d27848]) successfully deleted
I20260808 07:00:21.251547 27798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:00:21.251776 27770 catalog_manager.cc:4841] TS 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759): tablet c4e6b9ba792c437cb5edca7b8aa267ba (table test-table [id=d3d854eb320b435cbc8a037d91d27848]) successfully deleted
W20260808 07:00:21.251809 27798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:21.251850 27776 catalog_manager.cc:4841] TS 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027): tablet c4e6b9ba792c437cb5edca7b8aa267ba (table test-table [id=d3d854eb320b435cbc8a037d91d27848]) successfully deleted
I20260808 07:00:21.253183 28006 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 (DEFAULT_TABLE table=test-table [id=45f817c9d3a2484ab3aefc70979bc4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.253188 28418 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 (DEFAULT_TABLE table=test-table [id=45f817c9d3a2484ab3aefc70979bc4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.253324 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c3c2a190149c9ab8bf46a704c4998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.253352 28418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c3c2a190149c9ab8bf46a704c4998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.254236 28692 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Bootstrap starting.
I20260808 07:00:21.254307 28222 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 (DEFAULT_TABLE table=test-table [id=45f817c9d3a2484ab3aefc70979bc4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:21.254449 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c3c2a190149c9ab8bf46a704c4998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:21.254642 28692 tablet_bootstrap.cc:654] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.254971 28693 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: Bootstrap starting.
I20260808 07:00:21.255316 28693 tablet_bootstrap.cc:654] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.255649 28691 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: Bootstrap starting.
I20260808 07:00:21.255692 28692 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: No bootstrap required, opened a new log
I20260808 07:00:21.255769 28692 ts_tablet_manager.cc:1397] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:21.255896 28692 raft_consensus.cc:348] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.255982 28692 raft_consensus.cc:374] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.255999 28691 tablet_bootstrap.cc:654] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:21.256012 28692 raft_consensus.cc:729] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.256062 28692 consensus_queue.cc:260] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.256158 28692 ts_tablet_manager.cc:1428] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:21.256372 28693 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: No bootstrap required, opened a new log
I20260808 07:00:21.256433 28693 ts_tablet_manager.cc:1397] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:21.256565 28691 tablet_bootstrap.cc:492] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: No bootstrap required, opened a new log
I20260808 07:00:21.256551 28693 raft_consensus.cc:348] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.256620 28693 raft_consensus.cc:374] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.256618 28691 ts_tablet_manager.cc:1397] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:21.256647 28693 raft_consensus.cc:729] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8b3663a4a4501a409879ab1e603ce, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.256695 28693 consensus_queue.cc:260] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.256726 28691 raft_consensus.cc:348] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.256794 28691 raft_consensus.cc:374] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:21.256821 28691 raft_consensus.cc:729] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18d51b20a2c444f2a6f4750e5944a50b, State: Initialized, Role: FOLLOWER
I20260808 07:00:21.256860 28693 ts_tablet_manager.cc:1428] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:21.256865 28691 consensus_queue.cc:260] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.256955 28691 ts_tablet_manager.cc:1428] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:21.335574 28661 raft_consensus.cc:482] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:21.335685 28661 raft_consensus.cc:504] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.335897 28661 leader_election.cc:288] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027)
I20260808 07:00:21.336153 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c3c2a190149c9ab8bf46a704c4998" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b" is_pre_election: true
I20260808 07:00:21.336153 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c3c2a190149c9ab8bf46a704c4998" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce" is_pre_election: true
I20260808 07:00:21.336246 28249 raft_consensus.cc:2454] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 0.
I20260808 07:00:21.336272 28038 raft_consensus.cc:2454] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 0.
I20260808 07:00:21.336434 28350 leader_election.cc:302] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18d51b20a2c444f2a6f4750e5944a50b, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:21.336565 28661 raft_consensus.cc:2781] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:21.336616 28661 raft_consensus.cc:482] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:21.336632 28661 raft_consensus.cc:3037] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.337392 28661 raft_consensus.cc:504] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.337528 28661 leader_election.cc:288] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759), 18d51b20a2c444f2a6f4750e5944a50b (127.24.181.66:36027)
I20260808 07:00:21.337734 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c3c2a190149c9ab8bf46a704c4998" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf8b3663a4a4501a409879ab1e603ce"
I20260808 07:00:21.337770 28249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c3c2a190149c9ab8bf46a704c4998" candidate_uuid: "500678191d4f4a168ceea97086bb1ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18d51b20a2c444f2a6f4750e5944a50b"
I20260808 07:00:21.337829 28038 raft_consensus.cc:3037] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.337842 28249 raft_consensus.cc:3037] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:21.338656 28038 raft_consensus.cc:2454] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 1.
I20260808 07:00:21.338656 28249 raft_consensus.cc:2454] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500678191d4f4a168ceea97086bb1ad0 in term 1.
I20260808 07:00:21.338841 28346 leader_election.cc:302] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf8b3663a4a4501a409879ab1e603ce, 500678191d4f4a168ceea97086bb1ad0; no voters:
I20260808 07:00:21.338990 28661 raft_consensus.cc:2781] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:21.339059 28661 raft_consensus.cc:686] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 1 LEADER]: Becoming Leader. State: Replica: 500678191d4f4a168ceea97086bb1ad0, State: Running, Role: LEADER
I20260808 07:00:21.339113 28661 consensus_queue.cc:237] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } }
I20260808 07:00:21.339726 27798 catalog_manager.cc:5495] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "500678191d4f4a168ceea97086bb1ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500678191d4f4a168ceea97086bb1ad0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37849 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:21.351115 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:21.349009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.24.181.67:37849)
0808 07:00:21.349167 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:21.349169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:21.349195 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:00:21.349214 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:21.349214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:21.349224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:21.349283 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:21.349422 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.349890 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.350546 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.350723 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.350772 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:21.350887 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:21.350953 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:21.351003 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:21.351041 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 07:00:21.351059 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260808 07:00:21.355769 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:21.353386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:45605 (local address 127.24.181.67:37849)
0808 07:00:21.353418 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:21.353419 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:21.353489 (+ 70us) server_negotiation.cc:407] Connection header received
0808 07:00:21.353583 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:21.353584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:21.353592 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:21.353634 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:21.353766 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.354284 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.355126 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.355391 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.355447 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:21.355567 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:21.355591 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:21.355631 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:21.355705 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:00:21.355722 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 07:00:21.355901 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:21.353264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37849 (local address 127.24.181.67:45605)
0808 07:00:21.353433 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:21.353441 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:21.353525 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:21.353647 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:21.353647 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:21.353649 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:21.353684 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:21.353685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.354344 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.354345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:21.355005 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:21.355007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:21.355456 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:21.355457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:21.355498 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:21.355502 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:21.355642 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:21.355643 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:21.355729 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:00:21.355766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260808 07:00:21.357642 28038 raft_consensus.cc:1261] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce [term 1 FOLLOWER]: Refusing update from remote peer 500678191d4f4a168ceea97086bb1ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:21.357671 28248 raft_consensus.cc:1261] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Refusing update from remote peer 500678191d4f4a168ceea97086bb1ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:21.357964 28661 consensus_queue.cc:1035] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf8b3663a4a4501a409879ab1e603ce" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:21.358069 28651 consensus_queue.cc:1035] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18d51b20a2c444f2a6f4750e5944a50b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:00:21.362387 28641 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:00:22.884353 27798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:22.884472 27798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:22.885334 27798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6f144bae154b2c973c857df9dcf8ba: Sending DeleteTablet for 3 replicas of tablet ff5c3c2a190149c9ab8bf46a704c4998
I20260808 07:00:22.885591 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:22 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:48348
I20260808 07:00:22.885612 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:22 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55706
I20260808 07:00:22.885870 28418 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c3c2a190149c9ab8bf46a704c4998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:22 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40670
I20260808 07:00:22.885895 28753 tablet_replica.cc:330] T ff5c3c2a190149c9ab8bf46a704c4998 P 0bf8b3663a4a4501a409879ab1e603ce: stopping tablet replica
I20260808 07:00:22.885984 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 27905
I20260808 07:00:22.892565 28755 tablet_replica.cc:330] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: stopping tablet replica
I20260808 07:00:22.892716 28755 raft_consensus.cc:2229] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:22.892895 28755 raft_consensus.cc:2258] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:22.893356 28755 ts_tablet_manager.cc:1905] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:22.894271 28754 tablet_replica.cc:330] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: stopping tablet replica
I20260808 07:00:22.894359 28754 raft_consensus.cc:2229] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:22.894415 28754 raft_consensus.cc:2258] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:22.895352 28755 ts_tablet_manager.cc:1918] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:00:22.895428 28755 log.cc:1196] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/ff5c3c2a190149c9ab8bf46a704c4998
I20260808 07:00:22.895702 28755 ts_tablet_manager.cc:1939] T ff5c3c2a190149c9ab8bf46a704c4998 P 500678191d4f4a168ceea97086bb1ad0: Deleting consensus metadata
I20260808 07:00:22.896066 27776 catalog_manager.cc:4841] TS 500678191d4f4a168ceea97086bb1ad0 (127.24.181.67:37849): tablet ff5c3c2a190149c9ab8bf46a704c4998 (table test-table [id=45f817c9d3a2484ab3aefc70979bc4fc]) successfully deleted
I20260808 07:00:22.900751 28754 ts_tablet_manager.cc:1905] T ff5c3c2a190149c9ab8bf46a704c4998 P 18d51b20a2c444f2a6f4750e5944a50b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:00:22.900897 27770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:40759 (error 108)
I20260808 07:00:22.901244 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 28129
I20260808 07:00:22.906884 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:22.901319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:40759 (local address 127.0.0.1:48366)
0808 07:00:22.906783 (+ 5464us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:22.906848 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:40759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5437,"thread_start_us":66,"threads_started":1}
W20260808 07:00:22.907371 27770 catalog_manager.cc:4584] TS 0bf8b3663a4a4501a409879ab1e603ce (127.24.181.65:40759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff5c3c2a190149c9ab8bf46a704c4998: Network error: Client connection negotiation failed: client connection to 127.24.181.65:40759: connect: Connection refused (error 111)
I20260808 07:00:22.912868 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:22.912718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36027 (local address 127.0.0.1:55722)
0808 07:00:22.912796 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:22.912832 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.66:36027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:00:22.912863 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 28329
I20260808 07:00:22.923288 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 27751
2026-08-08T07:00:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7744 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:22 dist-test-slave-nfb5 krb5kdc[28762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:22 dist-test-slave-nfb5 krb5kdc[28762](info): set up 2 sockets
Aug 08 07:00:22 dist-test-slave-nfb5 krb5kdc[28762](info): commencing operation
krb5kdc: starting...
W20260808 07:00:24.135967 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:24.257931 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33121
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:33121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:24.338990 28781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:24.339191 28781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:24.339213 28781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:24.340922 28781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:24.340971 28781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:24.340986 28781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:24.340999 28781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:24.341012 28781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:24.342500 28781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:33121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:24.342777 28781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:24.343065 28781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:24.348263 28787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.348289 28791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.348315 28781 server_base.cc:1034] running on GCE node
W20260808 07:00:24.348316 28788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.348790 28781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:24.349079 28781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:24.350257 28781 hybrid_clock.cc:648] HybridClock initialized: now 1786172424350245 us; error 34 us; skew 500 ppm
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:24.355994 28781 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:24.356534 28781 webserver.cc:460] Webserver started at http://127.24.181.126:36355/ using document root <none> and password file <none>
I20260808 07:00:24.356783 28781 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:24.356837 28781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:24.357031 28781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:24.358189 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "31bcbc4e82974933a385c57fc802640d"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "90ff00b2d57c00d7edab8107fbccf064"
server_key_iv: "3c5a888ecd1dcb706a6f7b69327a1dc1"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.358600 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "31bcbc4e82974933a385c57fc802640d"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "90ff00b2d57c00d7edab8107fbccf064"
server_key_iv: "3c5a888ecd1dcb706a6f7b69327a1dc1"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.360163 28781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:24.360996 28799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.361191 28781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:24.361295 28781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "31bcbc4e82974933a385c57fc802640d"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "90ff00b2d57c00d7edab8107fbccf064"
server_key_iv: "3c5a888ecd1dcb706a6f7b69327a1dc1"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.361392 28781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:24.371268 28781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:24.373751 28781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:24.373917 28781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:24.378474 28781 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:33121
I20260808 07:00:24.378506 28880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:33121 every 8 connection(s)
I20260808 07:00:24.378875 28781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:24.379789 28882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:24.381964 28882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d: Bootstrap starting.
I20260808 07:00:24.382511 28882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:24.382645 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 28781
I20260808 07:00:24.382761 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:24.382889 28882 log.cc:824] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:24.382982 25301 external_mini_cluster.cc:1428] Setting key bad52a98ff562afdc781ab2dd1e6da4e
I20260808 07:00:24.383711 28882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d: No bootstrap required, opened a new log
I20260808 07:00:24.385758 28882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } }
I20260808 07:00:24.385923 28882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:24.385972 28882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31bcbc4e82974933a385c57fc802640d, State: Initialized, Role: FOLLOWER
I20260808 07:00:24.386104 28882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } }
I20260808 07:00:24.386178 28882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:24.386202 28882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:24.386265 28882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:24.386854 28882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } }
I20260808 07:00:24.387003 28882 leader_election.cc:302] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31bcbc4e82974933a385c57fc802640d; no voters:
I20260808 07:00:24.387174 28882 leader_election.cc:288] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:24.387234 28889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:24.387391 28889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [term 1 LEADER]: Becoming Leader. State: Replica: 31bcbc4e82974933a385c57fc802640d, State: Running, Role: LEADER
I20260808 07:00:24.387436 28882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:24.387553 28889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } }
I20260808 07:00:24.388147 28890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31bcbc4e82974933a385c57fc802640d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } } }
I20260808 07:00:24.388438 28890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:24.388352 28891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31bcbc4e82974933a385c57fc802640d. Latest consensus state: current_term: 1 leader_uuid: "31bcbc4e82974933a385c57fc802640d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bcbc4e82974933a385c57fc802640d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33121 } } }
I20260808 07:00:24.388667 28891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:24.388684 28900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:24.389292 28900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:24.391667 28900 catalog_manager.cc:1347] Generated new cluster ID: 7e49a3ad5c4a4a21977c41f8b8b6b588
I20260808 07:00:24.391722 28900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:24.394654 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.383466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.24.181.126:33121)
0808 07:00:24.383656 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:24.383666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:24.383700 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:00:24.384129 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:24.384135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:24.384179 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:24.384325 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:24.384979 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.385855 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.386953 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.387581 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.387707 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.390736 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:24.390740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:24.390740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:24.390752 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:24.393008 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.393421 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.393424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.393424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.393537 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.393747 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.393748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.393749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.394057 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:24.394225 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:24.394300 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:00:24.394441 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260808 07:00:24.412592 28900 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:24.413115 28900 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:24.417858 28900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31bcbc4e82974933a385c57fc802640d: Generated new TSK 0
I20260808 07:00:24.418068 28900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:24.449954 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:24.532995 28919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:24.533202 28919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:24.533227 28919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:24.534391 28919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:24.534436 28919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:24.534463 28919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:24.535998 28919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:24.536254 28919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:24.536484 28919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:24.541572 28925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.541536 28926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.541600 28919 server_base.cc:1034] running on GCE node
W20260808 07:00:24.541548 28930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.542075 28919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:24.542291 28919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:24.543435 28919 hybrid_clock.cc:648] HybridClock initialized: now 1786172424543409 us; error 46 us; skew 500 ppm
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:24.546054 28919 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:24.546509 28919 webserver.cc:460] Webserver started at http://127.24.181.65:42981/ using document root <none> and password file <none>
I20260808 07:00:24.546743 28919 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:24.546789 28919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:24.546983 28919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:24.548019 28919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "bc0fb5517b854128930a4c6b9a650041"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d65736b4a16944ea538100e570dfe5f2"
server_key_iv: "c4ddc5e5bd86b15653876c26c5b0e1ce"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.548382 28919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "bc0fb5517b854128930a4c6b9a650041"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d65736b4a16944ea538100e570dfe5f2"
server_key_iv: "c4ddc5e5bd86b15653876c26c5b0e1ce"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.549731 28919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:24.550464 28939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.550642 28919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:24.550742 28919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "bc0fb5517b854128930a4c6b9a650041"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d65736b4a16944ea538100e570dfe5f2"
server_key_iv: "c4ddc5e5bd86b15653876c26c5b0e1ce"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.550827 28919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:24.565981 28919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:24.568039 28919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:24.568218 28919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:24.568454 28919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:24.568776 28919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:24.568835 28919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.568871 28919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:24.568897 28919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.574680 28919 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36475
I20260808 07:00:24.574755 29102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36475 every 8 connection(s)
I20260808 07:00:24.575100 28919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:24.575417 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 28919
I20260808 07:00:24.575519 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:24.575711 25301 external_mini_cluster.cc:1428] Setting key fc7d1c9e8b436ec079ab2acf5af5cfd8
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:00:24.585543 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.575915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:45087 (local address 127.24.181.126:33121)
0808 07:00:24.575994 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:24.575997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:24.576224 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:00:24.576782 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:24.576783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:24.576797 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:24.576860 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:24.577763 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.578403 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.579800 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.580022 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.580082 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.582512 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:24.582513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:24.582514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:24.582521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:24.584016 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.584456 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.584458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.584458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.584521 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.584823 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.584823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.584824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.584955 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:24.585033 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:24.585312 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 07:00:24.585404 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:00:24.585677 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.575852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33121 (local address 127.24.181.65:45087)
0808 07:00:24.576084 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:24.576131 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:24.576673 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:24.576919 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:24.576923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:24.577493 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:24.577670 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.577674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.578512 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.578513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.579689 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.579692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.580128 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.580129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.580293 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.580799 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:24.580812 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:24.582411 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:24.584123 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.584131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.584134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.584374 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.584611 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.584612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.584612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.584741 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.585026 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:24.585034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:24.585174 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:00:24.585279 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:00:24.585447 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
I20260808 07:00:24.586190 29105 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33121
I20260808 07:00:24.586287 29105 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:24.586503 29105 heartbeater.cc:511] Master 127.24.181.126:33121 requested a full tablet report, sending...
I20260808 07:00:24.587136 28834 ts_manager.cc:194] Registered new tserver with Master: bc0fb5517b854128930a4c6b9a650041 (127.24.181.65:36475)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
I20260808 07:00:24.587953 28834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:45087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:24.602707 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:24.684751 29111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:24.685001 29111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:24.685039 29111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:24.686213 29111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:24.686283 29111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:24.686328 29111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:24.687947 29111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:24.688222 29111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:24.688493 29111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:24.693879 29122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.693940 29119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.693984 29118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.694125 29111 server_base.cc:1034] running on GCE node
I20260808 07:00:24.694357 29111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:24.694618 29111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:24.695793 29111 hybrid_clock.cc:648] HybridClock initialized: now 1786172424695765 us; error 47 us; skew 500 ppm
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:24.698459 29111 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:24.698843 29111 webserver.cc:460] Webserver started at http://127.24.181.66:45041/ using document root <none> and password file <none>
I20260808 07:00:24.699113 29111 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:24.699162 29111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:24.699251 29111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:24.700258 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "60f58ae2ab38454887bc071d2867da46"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d64b755a4c3529b1f1c7ccc486ae670b"
server_key_iv: "03fcb5ff46fa66587c8a8fb8c6de74d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.700577 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "60f58ae2ab38454887bc071d2867da46"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d64b755a4c3529b1f1c7ccc486ae670b"
server_key_iv: "03fcb5ff46fa66587c8a8fb8c6de74d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.701975 29111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:24.702759 29133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.703016 29111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:24.703126 29111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "60f58ae2ab38454887bc071d2867da46"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "d64b755a4c3529b1f1c7ccc486ae670b"
server_key_iv: "03fcb5ff46fa66587c8a8fb8c6de74d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.703219 29111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:24.720729 29111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:24.723095 29111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:24.723281 29111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:24.723548 29111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:24.723898 29111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:24.723958 29111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.724004 29111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:24.724033 29111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.730055 29111 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:35289
I20260808 07:00:24.730110 29292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:35289 every 8 connection(s)
I20260808 07:00:24.730458 29111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:24.737663 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 29111
I20260808 07:00:24.737746 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:24.737964 25301 external_mini_cluster.cc:1428] Setting key fc615f70661f039bdbede6eeac844d21
I20260808 07:00:24.740305 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.731297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:53327 (local address 127.24.181.126:33121)
0808 07:00:24.731383 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:24.731386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:24.731526 (+ 140us) server_negotiation.cc:407] Connection header received
0808 07:00:24.732001 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:24.732002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:24.732012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:24.732068 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:24.733045 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.733528 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.734536 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.734715 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.734766 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.737063 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:24.737064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:24.737064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:24.737072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:24.738686 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.739254 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.739255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.739255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.739313 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.739634 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.739635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.739635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.739753 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:24.739807 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:24.740166 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 07:00:24.740240 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:00:24.740622 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.731230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33121 (local address 127.24.181.66:53327)
0808 07:00:24.731400 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:24.731438 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:24.731919 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:24.732135 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:24.732138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:24.732766 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:24.732965 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.732969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.733610 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.733611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.734456 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.734459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.734790 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.734791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.734896 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.735505 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:24.735514 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:24.736964 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:24.738778 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.738787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.738789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.739181 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.739387 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.739388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.739388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.739546 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.739827 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:24.739847 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:24.740022 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:00:24.740144 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:00:24.740331 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260808 07:00:24.741175 29293 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33121
I20260808 07:00:24.741298 29293 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:24.741516 29293 heartbeater.cc:511] Master 127.24.181.126:33121 requested a full tablet report, sending...
I20260808 07:00:24.741972 28834 ts_manager.cc:194] Registered new tserver with Master: 60f58ae2ab38454887bc071d2867da46 (127.24.181.66:35289)
I20260808 07:00:24.742378 28834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:53327
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:24.766326 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:24.854986 29300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:24.855243 29300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:24.855283 29300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:24.856563 29300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:24.856645 29300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:24.856694 29300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:24.858300 29300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:24.858596 29300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:24.858846 29300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:00:24.864259 29300 server_base.cc:1034] running on GCE node
W20260808 07:00:24.864265 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.864379 29307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:24.864274 29314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:24.864748 29300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:24.865041 29300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:24.866254 29300 hybrid_clock.cc:648] HybridClock initialized: now 1786172424866240 us; error 27 us; skew 500 ppm
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:24.869069 29300 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:24.869575 29300 webserver.cc:460] Webserver started at http://127.24.181.67:45007/ using document root <none> and password file <none>
I20260808 07:00:24.869843 29300 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:24.869932 29300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:24.870092 29300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:24.871275 29300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "2711de8a012548c5a9414da58ccd94be"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "9aaeccae4e8010eef245b0913681cca3"
server_key_iv: "4a3399154bdff8102e3cd4725cab0b08"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.871678 29300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "2711de8a012548c5a9414da58ccd94be"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "9aaeccae4e8010eef245b0913681cca3"
server_key_iv: "4a3399154bdff8102e3cd4725cab0b08"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.873068 29300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:24.873840 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.874020 29300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:24.874133 29300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "2711de8a012548c5a9414da58ccd94be"
format_stamp: "Formatted at 2026-08-08 07:00:24 on dist-test-slave-nfb5"
server_key: "9aaeccae4e8010eef245b0913681cca3"
server_key_iv: "4a3399154bdff8102e3cd4725cab0b08"
server_key_version: "encryptionkey@0"
I20260808 07:00:24.874212 29300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:24.883179 29300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:24.885238 29300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:24.885412 29300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:24.885663 29300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:24.886006 29300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:24.886065 29300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.886111 29300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:24.886133 29300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:24.892753 29300 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:39793
I20260808 07:00:24.892799 29474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:39793 every 8 connection(s)
I20260808 07:00:24.893165 29300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:00:24 dist-test-slave-nfb5 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172424, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:24.902233 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 29300
I20260808 07:00:24.902345 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:24.902567 25301 external_mini_cluster.cc:1428] Setting key b084e68464aa3ac4d86f9abb1cabe689
I20260808 07:00:24.902678 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.893940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:46031 (local address 127.24.181.126:33121)
0808 07:00:24.894001 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:24.894003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:24.894239 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:00:24.894791 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:24.894793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:24.894807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:24.894860 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:24.895919 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.896446 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.897456 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.897664 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.897754 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.900079 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:24.900080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:24.900080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:24.900088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:24.901308 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.901706 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.901706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.901707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.901757 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:24.901977 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:24.901978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:24.901978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:24.902080 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:24.902133 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:24.902498 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 07:00:24.902566 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:00:24.902987 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:24.893915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33121 (local address 127.24.181.67:46031)
0808 07:00:24.894106 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:24.894143 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:24.894674 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:24.894997 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:24.895001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:24.895596 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:24.895797 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.895801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.896558 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.896559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.897359 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:24.897361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:24.897767 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:24.897768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:24.897899 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:24.898432 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:24.898440 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:24.899992 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:24.901398 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.901405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.901407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.901641 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.901818 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:24.901818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:24.901819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:24.901921 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:24.902155 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:24.902163 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:24.902347 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 07:00:24.902484 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 07:00:24.902671 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260808 07:00:24.903584 29476 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33121
I20260808 07:00:24.903716 29476 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:24.903928 29476 heartbeater.cc:511] Master 127.24.181.126:33121 requested a full tablet report, sending...
I20260808 07:00:24.904311 28833 ts_manager.cc:194] Registered new tserver with Master: 2711de8a012548c5a9414da58ccd94be (127.24.181.67:39793)
I20260808 07:00:24.904712 28833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:46031
I20260808 07:00:24.914726 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:00:24.928543 28888 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:24.927620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.24.181.126:33121)
0808 07:00:24.927681 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:24.927688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:24.927773 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:00:24.927823 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:24.927824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:24.927833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:24.927899 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:24.928482 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43358: BlockingRecv error: recv got EOF from 127.0.0.1:43358 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:00:24.929426 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 28919
I20260808 07:00:24.942301 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 29111
I20260808 07:00:24.958613 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 29300
I20260808 07:00:24.968981 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 28781
2026-08-08T07:00:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2048 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:25 dist-test-slave-nfb5 krb5kdc[29491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:25 dist-test-slave-nfb5 krb5kdc[29491](info): set up 2 sockets
Aug 08 07:00:25 dist-test-slave-nfb5 krb5kdc[29491](info): commencing operation
krb5kdc: starting...
W20260808 07:00:26.186168 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:26.309953 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43819
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:26.391515 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:26.391719 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:26.391741 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:26.392843 29510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:26.392879 29510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:26.392892 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:26.392905 29510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:26.392916 29510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:26.394362 29510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:26.394546 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:26.394719 29510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:26.399808 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:26.399854 29518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:26.399909 29510 server_base.cc:1034] running on GCE node
W20260808 07:00:26.399838 29517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:26.400235 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:26.400544 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:26.401690 29510 hybrid_clock.cc:648] HybridClock initialized: now 1786172426401676 us; error 27 us; skew 500 ppm
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172426, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:26.404271 29510 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:26.404753 29510 webserver.cc:460] Webserver started at http://127.24.181.126:34433/ using document root <none> and password file <none>
I20260808 07:00:26.405001 29510 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:26.405077 29510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:26.405233 29510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:26.406275 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "da9b2b9827f74e568b7eee697d662346"
format_stamp: "Formatted at 2026-08-08 07:00:26 on dist-test-slave-nfb5"
server_key: "3d740b3ef90e2dec8bd19ed3927578cd"
server_key_iv: "c51c5282018a4cfad1c7cd3f58467ae6"
server_key_version: "encryptionkey@0"
I20260808 07:00:26.406635 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "da9b2b9827f74e568b7eee697d662346"
format_stamp: "Formatted at 2026-08-08 07:00:26 on dist-test-slave-nfb5"
server_key: "3d740b3ef90e2dec8bd19ed3927578cd"
server_key_iv: "c51c5282018a4cfad1c7cd3f58467ae6"
server_key_version: "encryptionkey@0"
I20260808 07:00:26.408056 29510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:26.408769 29531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:26.408955 29510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:26.409058 29510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "da9b2b9827f74e568b7eee697d662346"
format_stamp: "Formatted at 2026-08-08 07:00:26 on dist-test-slave-nfb5"
server_key: "3d740b3ef90e2dec8bd19ed3927578cd"
server_key_iv: "c51c5282018a4cfad1c7cd3f58467ae6"
server_key_version: "encryptionkey@0"
I20260808 07:00:26.409142 29510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:26.433295 29510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:26.435236 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:26.435413 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:26.439649 29510 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:43819
I20260808 07:00:26.439690 29605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:43819 every 8 connection(s)
I20260808 07:00:26.440264 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:26.440955 29606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:26.443293 29606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346: Bootstrap starting.
I20260808 07:00:26.443967 29606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:26.444218 29606 log.cc:824] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:26.444917 29606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346: No bootstrap required, opened a new log
I20260808 07:00:26.445102 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 29510
I20260808 07:00:26.445204 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:26.445390 25301 external_mini_cluster.cc:1428] Setting key 175e2114d32407c6a1fbb4f9b85f52e7
I20260808 07:00:26.448104 29606 raft_consensus.cc:348] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } }
I20260808 07:00:26.448422 29606 raft_consensus.cc:374] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:26.448493 29606 raft_consensus.cc:729] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9b2b9827f74e568b7eee697d662346, State: Initialized, Role: FOLLOWER
I20260808 07:00:26.448634 29606 consensus_queue.cc:260] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } }
I20260808 07:00:26.448805 29606 raft_consensus.cc:388] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:26.448846 29606 raft_consensus.cc:482] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:26.448905 29606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:26.449536 29606 raft_consensus.cc:504] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } }
I20260808 07:00:26.449674 29606 leader_election.cc:302] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da9b2b9827f74e568b7eee697d662346; no voters:
I20260808 07:00:26.449872 29606 leader_election.cc:288] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:26.449934 29612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:26.450065 29612 raft_consensus.cc:686] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [term 1 LEADER]: Becoming Leader. State: Replica: da9b2b9827f74e568b7eee697d662346, State: Running, Role: LEADER
I20260808 07:00:26.450145 29606 sys_catalog.cc:564] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:26.450150 29612 consensus_queue.cc:237] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } }
I20260808 07:00:26.450532 29614 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da9b2b9827f74e568b7eee697d662346. Latest consensus state: current_term: 1 leader_uuid: "da9b2b9827f74e568b7eee697d662346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } } }
I20260808 07:00:26.450616 29614 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:26.451247 29619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:26.451768 29619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:26.451871 29613 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da9b2b9827f74e568b7eee697d662346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9b2b9827f74e568b7eee697d662346" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43819 } } }
I20260808 07:00:26.452036 29613 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:26.455035 29619 catalog_manager.cc:1347] Generated new cluster ID: 4821e6a56d6341048b01dc8ad27bd2a2
I20260808 07:00:26.455111 29619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:26.458696 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:26.445949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.24.181.126:43819)
0808 07:00:26.446177 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:26.446186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:26.446220 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:00:26.448012 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:26.448024 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:26.448099 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:26.448255 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:26.449163 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:26.450307 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:26.451555 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:26.452160 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:26.452273 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:26.454770 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:26.454773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:26.454774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:26.454785 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:26.456880 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:26.457280 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:26.457283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:26.457284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:26.457405 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:26.457616 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:26.457617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:26.457617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:26.457923 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:26.458094 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:26.458177 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:00:26.458340 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":110,"threads_started":1}
I20260808 07:00:26.459571 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:00:26.471238 29619 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:26.471783 29619 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:26.476490 29619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da9b2b9827f74e568b7eee697d662346: Generated new TSK 0
I20260808 07:00:26.476683 29619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:26.480581 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:26.477083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.24.181.126:43819)
0808 07:00:26.477122 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:26.477125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:26.477266 (+ 141us) server_negotiation.cc:407] Connection header received
0808 07:00:26.477337 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:26.477338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:26.477349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:26.477409 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:26.477805 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:26.478291 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:26.479096 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:26.479327 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:26.479405 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:26.480123 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:26.480124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:26.480125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:26.480310 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:26.480326 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:26.480332 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:26.480403 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:26.480470 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:00:26.480496 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:00:26.483276 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 29510
2026-08-08T07:00:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1510 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29647](info): set up 2 sockets
Aug 08 07:00:26 dist-test-slave-nfb5 krb5kdc[29647](info): commencing operation
krb5kdc: starting...
W20260808 07:00:27.704348 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:27 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:27.823643 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:42073
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:42073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:27.904603 29665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:27.904805 29665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:27.904827 29665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:27.905978 29665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:27.906021 29665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:27.906036 29665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:27.906049 29665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:27.906061 29665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:27.907527 29665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:42073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:42073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:27.907759 29665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:27.907945 29665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:27.912788 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:27.912837 29672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:27.912847 29671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:27.912948 29665 server_base.cc:1034] running on GCE node
I20260808 07:00:27.913262 29665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:27.913556 29665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:27.914738 29665 hybrid_clock.cc:648] HybridClock initialized: now 1786172427914721 us; error 36 us; skew 500 ppm
Aug 08 07:00:27 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:27.917305 29665 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:27.917676 29665 webserver.cc:460] Webserver started at http://127.24.181.126:34293/ using document root <none> and password file <none>
I20260808 07:00:27.917903 29665 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:27.917955 29665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:27.918045 29665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:27.919106 29665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "4dd7c641198847e1b63b94148036b308"
format_stamp: "Formatted at 2026-08-08 07:00:27 on dist-test-slave-nfb5"
server_key: "c942af4223ad9dda89d665d563b7ff7f"
server_key_iv: "195dc63b283860a7cb7de8c70486eeb5"
server_key_version: "encryptionkey@0"
I20260808 07:00:27.919404 29665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "4dd7c641198847e1b63b94148036b308"
format_stamp: "Formatted at 2026-08-08 07:00:27 on dist-test-slave-nfb5"
server_key: "c942af4223ad9dda89d665d563b7ff7f"
server_key_iv: "195dc63b283860a7cb7de8c70486eeb5"
server_key_version: "encryptionkey@0"
I20260808 07:00:27.920650 29665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:27.921377 29688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:27.921574 29665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:27.921662 29665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "4dd7c641198847e1b63b94148036b308"
format_stamp: "Formatted at 2026-08-08 07:00:27 on dist-test-slave-nfb5"
server_key: "c942af4223ad9dda89d665d563b7ff7f"
server_key_iv: "195dc63b283860a7cb7de8c70486eeb5"
server_key_version: "encryptionkey@0"
I20260808 07:00:27.921716 29665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:27.943573 29665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:27.945894 29665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:27.946005 29665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:27.950115 29665 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:42073
I20260808 07:00:27.950246 29759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:42073 every 8 connection(s)
I20260808 07:00:27.950544 29665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:27.951630 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:27.953979 29760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308: Bootstrap starting.
I20260808 07:00:27.954576 29760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:27.954873 29760 log.cc:824] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:27.955672 29760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308: No bootstrap required, opened a new log
I20260808 07:00:27.957285 29760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } }
I20260808 07:00:27.957424 29760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:27.957484 29760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd7c641198847e1b63b94148036b308, State: Initialized, Role: FOLLOWER
I20260808 07:00:27.957717 29760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } }
I20260808 07:00:27.957793 29760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:27.957844 29760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:27.957918 29760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:27.958425 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 29665
I20260808 07:00:27.958546 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:27.958740 25301 external_mini_cluster.cc:1428] Setting key e36885680987b7f0a3fc4fff499dd555
I20260808 07:00:27.958876 29760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } }
I20260808 07:00:27.959033 29760 leader_election.cc:302] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd7c641198847e1b63b94148036b308; no voters:
I20260808 07:00:27.959228 29760 leader_election.cc:288] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:27.959254 29763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:27.959563 29760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:27.959683 29763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd7c641198847e1b63b94148036b308, State: Running, Role: LEADER
I20260808 07:00:27.959824 29763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } }
I20260808 07:00:27.960175 29765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dd7c641198847e1b63b94148036b308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } } }
I20260808 07:00:27.960201 29769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dd7c641198847e1b63b94148036b308. Latest consensus state: current_term: 1 leader_uuid: "4dd7c641198847e1b63b94148036b308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7c641198847e1b63b94148036b308" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42073 } } }
I20260808 07:00:27.960265 29765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:27.960290 29769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:27.960472 29778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:27.961030 29778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:27.962863 29778 catalog_manager.cc:1347] Generated new cluster ID: e8d916abf24c497aa1edcf5e7efa1649
I20260808 07:00:27.962934 29778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:27 dist-test-slave-nfb5 krb5kdc[29647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:27.970732 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:27.959463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39020 (local address 127.24.181.126:42073)
0808 07:00:27.959631 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:27.959640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:27.959672 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:00:27.959786 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:27.959792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:27.959850 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:27.960018 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:27.960908 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:27.962347 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:27.963465 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:27.964077 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:27.964187 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:27.967001 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:27.967005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:27.967005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:27.967017 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:27.969085 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:27.969481 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:27.969485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:27.969485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:27.969607 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:27.969817 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:27.969818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:27.969818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:27.970125 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:27.970290 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:27.970380 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:00:27.970524 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260808 07:00:27.974215 29778 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:27.974709 29778 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:27.984565 29778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dd7c641198847e1b63b94148036b308: Generated new TSK 0
I20260808 07:00:27.984834 29778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:28.014391 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:28.106074 29797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:28.106328 29797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:28.106369 29797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:28.107585 29797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:28.107650 29797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:28.107699 29797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:28.109216 29797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:28.109483 29797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:28.109776 29797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:28.114945 29804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:28.114969 29805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:28.115015 29797 server_base.cc:1034] running on GCE node
W20260808 07:00:28.115078 29808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:28.115350 29797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:28.115604 29797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:28.116788 29797 hybrid_clock.cc:648] HybridClock initialized: now 1786172428116793 us; error 70 us; skew 500 ppm
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:28.119441 29797 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:28.119880 29797 webserver.cc:460] Webserver started at http://127.24.181.65:45899/ using document root <none> and password file <none>
I20260808 07:00:28.120131 29797 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:28.120178 29797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:28.120347 29797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:28.121415 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "44fa2ce881dd449d8296237f945359e9"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "a0c6d86edc5b9d4ef2c961861cd4f2c1"
server_key_iv: "50113bdda52ad7427f3b64993eb9d19b"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.121775 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "44fa2ce881dd449d8296237f945359e9"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "a0c6d86edc5b9d4ef2c961861cd4f2c1"
server_key_iv: "50113bdda52ad7427f3b64993eb9d19b"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.123173 29797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:00:28.123957 29817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.124181 29797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:28.124285 29797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "44fa2ce881dd449d8296237f945359e9"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "a0c6d86edc5b9d4ef2c961861cd4f2c1"
server_key_iv: "50113bdda52ad7427f3b64993eb9d19b"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.124390 29797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:28.151356 29797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:28.153375 29797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:28.153543 29797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:28.153785 29797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:28.154102 29797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:28.154160 29797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.154207 29797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:28.154242 29797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.160233 29797 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:32979
I20260808 07:00:28.160305 29993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:32979 every 8 connection(s)
I20260808 07:00:28.160676 29797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:28.170210 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.161500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:34973 (local address 127.24.181.126:42073)
0808 07:00:28.161547 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:28.161549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:28.161810 (+ 261us) server_negotiation.cc:407] Connection header received
0808 07:00:28.162379 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:28.162381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:28.162397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:28.162447 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:28.163457 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.163974 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.165008 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.165231 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.165315 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.167480 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:28.167482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:28.167482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:28.167490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:28.168783 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.169193 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.169194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.169195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.169261 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.169516 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.169517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.169518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.169649 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:28.169741 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:28.170008 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:00:28.170101 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:28.170413 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.161484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42073 (local address 127.24.181.65:34973)
0808 07:00:28.161676 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:28.161715 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:28.162262 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:28.162526 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:28.162530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:28.163150 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:28.163348 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.163353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.164092 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.164093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.164909 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.164911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.165336 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.165337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.165470 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.165936 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:28.165945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:28.167374 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:28.168875 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.168882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.168885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.169112 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.169333 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.169334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.169334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.169444 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.169734 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:28.169742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:28.169877 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:00:28.169986 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:00:28.170140 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260808 07:00:28.170693 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 29797
I20260808 07:00:28.170825 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:28.171051 25301 external_mini_cluster.cc:1428] Setting key 8aecf244f671b764d8e34bac36fed8eb
I20260808 07:00:28.171043 29995 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42073
I20260808 07:00:28.171152 29995 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:28.171370 29995 heartbeater.cc:511] Master 127.24.181.126:42073 requested a full tablet report, sending...
I20260808 07:00:28.171923 29712 ts_manager.cc:194] Registered new tserver with Master: 44fa2ce881dd449d8296237f945359e9 (127.24.181.65:32979)
I20260808 07:00:28.172535 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:34973
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:28.197325 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:28.280050 30006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:28.280246 30006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:28.280269 30006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:28.281344 30006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:28.281381 30006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:28.281406 30006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:28.282815 30006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:28.283071 30006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:28.283265 30006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:28.288213 30021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:28.288229 30017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:28.288254 30015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:28.288492 30006 server_base.cc:1034] running on GCE node
I20260808 07:00:28.288741 30006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:28.288987 30006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:28.290156 30006 hybrid_clock.cc:648] HybridClock initialized: now 1786172428290140 us; error 30 us; skew 500 ppm
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:28.292842 30006 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:28.293342 30006 webserver.cc:460] Webserver started at http://127.24.181.66:40549/ using document root <none> and password file <none>
I20260808 07:00:28.293593 30006 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:28.293663 30006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:28.293818 30006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:28.294878 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "8c39056af8c84001ba4e443141ccb2a2"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "82258a93bfea179d5826423a4c3f377a"
server_key_iv: "3aa15ae5f440d2ec3838844a3764c06f"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.295315 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "8c39056af8c84001ba4e443141ccb2a2"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "82258a93bfea179d5826423a4c3f377a"
server_key_iv: "3aa15ae5f440d2ec3838844a3764c06f"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.296639 30006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:28.297375 30031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.297547 30006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:28.297653 30006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "8c39056af8c84001ba4e443141ccb2a2"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "82258a93bfea179d5826423a4c3f377a"
server_key_iv: "3aa15ae5f440d2ec3838844a3764c06f"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.297726 30006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:28.308543 30006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:28.310765 30006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:28.310986 30006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:28.311265 30006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:28.311582 30006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:28.311654 30006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.311700 30006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:28.311733 30006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.317864 30006 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:37735
I20260808 07:00:28.317912 30192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:37735 every 8 connection(s)
I20260808 07:00:28.318238 30006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:28.322152 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30006
I20260808 07:00:28.322280 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:28.322497 25301 external_mini_cluster.cc:1428] Setting key a80fa0b995c03db7720c681066151d50
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:28.328682 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.318955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:57403 (local address 127.24.181.126:42073)
0808 07:00:28.319019 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:28.319022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:28.319212 (+ 190us) server_negotiation.cc:407] Connection header received
0808 07:00:28.319724 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:28.319726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:28.319737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:28.319791 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:28.320729 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.321247 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.322414 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.322677 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.322757 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.325544 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:28.325545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:28.325545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:28.325551 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:28.327198 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.327579 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.327580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.327581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.327653 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.327968 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.327969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.327969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.328132 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:28.328219 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:28.328469 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 07:00:28.328566 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:00:28.328707 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.318885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42073 (local address 127.24.181.66:57403)
0808 07:00:28.319092 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:28.319128 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:28.319595 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:28.319843 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:28.319846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:28.320403 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:28.320585 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.320589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.321328 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.321329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.322326 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.322329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.322801 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.322802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.322989 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.323570 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:28.323581 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:28.325449 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:28.327274 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.327281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.327282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.327499 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.327774 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.327775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.327775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.327886 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.328199 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:28.328205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:28.328335 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 07:00:28.328421 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:00:28.328544 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":132,"threads_started":1}
I20260808 07:00:28.329380 30193 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42073
I20260808 07:00:28.329488 30193 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:28.329692 30193 heartbeater.cc:511] Master 127.24.181.126:42073 requested a full tablet report, sending...
I20260808 07:00:28.330161 29712 ts_manager.cc:194] Registered new tserver with Master: 8c39056af8c84001ba4e443141ccb2a2 (127.24.181.66:37735)
I20260808 07:00:28.330540 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:57403
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:28.350514 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:28.432830 30198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:28.433040 30198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:28.433064 30198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:28.434237 30198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:28.434281 30198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:28.434309 30198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:28.435875 30198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:28.436132 30198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:28.436378 30198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:28.441552 30207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:28.441552 30204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:28.441563 30203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:28.441870 30198 server_base.cc:1034] running on GCE node
I20260808 07:00:28.442070 30198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:28.442327 30198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:28.443498 30198 hybrid_clock.cc:648] HybridClock initialized: now 1786172428443483 us; error 46 us; skew 500 ppm
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:28.445994 30198 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:28.446550 30198 webserver.cc:460] Webserver started at http://127.24.181.67:37449/ using document root <none> and password file <none>
I20260808 07:00:28.446780 30198 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:28.446854 30198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:28.447029 30198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:28.448107 30198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "2ac6c57fee6a480ea3d6a2cc9315884f"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "90e58e49f6afc7535856e2dce2718ca3"
server_key_iv: "80ffdda0978dffd2595656aa4bac6e50"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.448485 30198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "2ac6c57fee6a480ea3d6a2cc9315884f"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "90e58e49f6afc7535856e2dce2718ca3"
server_key_iv: "80ffdda0978dffd2595656aa4bac6e50"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.449874 30198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:28.450611 30217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.450827 30198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:28.450948 30198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "2ac6c57fee6a480ea3d6a2cc9315884f"
format_stamp: "Formatted at 2026-08-08 07:00:28 on dist-test-slave-nfb5"
server_key: "90e58e49f6afc7535856e2dce2718ca3"
server_key_iv: "80ffdda0978dffd2595656aa4bac6e50"
server_key_version: "encryptionkey@0"
I20260808 07:00:28.451036 30198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:28.467865 30198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:28.469709 30198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:28.469882 30198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:28.470124 30198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:28.470458 30198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:28.470518 30198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.470561 30198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:28.470589 30198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:28.476722 30198 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:39991
I20260808 07:00:28.476778 30365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:39991 every 8 connection(s)
I20260808 07:00:28.477097 30198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:28.485713 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30198
I20260808 07:00:28.485822 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:28.486038 25301 external_mini_cluster.cc:1428] Setting key bacfa463dc85ed79727cc8f6c85ba689
I20260808 07:00:28.486740 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.477868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:54715 (local address 127.24.181.126:42073)
0808 07:00:28.477938 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:28.477940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:28.478166 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:00:28.478657 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:28.478658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:28.478667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:28.478745 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:28.479745 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.480238 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.481211 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.481390 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.481442 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.483529 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:28.483530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:28.483531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:28.483538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:28.485158 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.485531 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.485532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.485532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.485601 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.485925 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.485927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.485927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.486080 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:28.486159 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:28.486545 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 07:00:28.486634 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:00:28.487047 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.477860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42073 (local address 127.24.181.67:54715)
0808 07:00:28.478030 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:28.478074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:28.478574 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:28.478797 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:28.478800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:28.479457 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:28.479652 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.479657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.480333 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.480333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.481134 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:28.481136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.481465 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.481466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:28.481567 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.482026 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:28.482035 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:28.483443 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:28.485239 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.485246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.485247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.485464 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.485677 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:28.485678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:28.485678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:28.485803 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:28.486201 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:28.486209 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:28.486392 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 07:00:28.486511 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:00:28.486746 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260808 07:00:28.487607 30366 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42073
I20260808 07:00:28.487686 30366 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:28.487826 30366 heartbeater.cc:511] Master 127.24.181.126:42073 requested a full tablet report, sending...
I20260808 07:00:28.488149 29712 ts_manager.cc:194] Registered new tserver with Master: 2ac6c57fee6a480ea3d6a2cc9315884f (127.24.181.67:39991)
I20260808 07:00:28.488525 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:54715
I20260808 07:00:28.498030 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[29647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172428, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:28.521396 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:28.514102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.24.181.126:42073)
0808 07:00:28.514179 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:28.514181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:28.514286 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:00:28.514369 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:28.514370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:28.514386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:28.514450 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:28.515096 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.515596 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.516356 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:28.516546 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:28.516621 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:28.519006 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:28.519008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:28.519008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:28.519015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:28.520185 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.520522 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.520524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.520524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.520579 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:28.520817 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:28.520818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:28.520818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:28.520971 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:28.521082 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:28.521189 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:00:28.521273 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 07:00:28.521638 29712 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39022
I20260808 07:00:28.522073 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 29797
I20260808 07:00:28.537339 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30006
I20260808 07:00:28.546589 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30198
I20260808 07:00:28.561802 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 29665
2026-08-08T07:00:28Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2079 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[30381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[30381](info): set up 2 sockets
Aug 08 07:00:28 dist-test-slave-nfb5 krb5kdc[30381](info): commencing operation
krb5kdc: starting...
W20260808 07:00:29.786111 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:29 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:29.905926 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:39515
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:39515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:29.991245 30402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:29.991556 30402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:29.991596 30402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:29.993234 30402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:29.993348 30402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:29.993388 30402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:29.993445 30402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:29.993477 30402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:29.995168 30402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:39515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:39515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:29.995450 30402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:29.995684 30402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:30.000610 30413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.000586 30415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.000588 30417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:30.000677 30402 server_base.cc:1034] running on GCE node
I20260808 07:00:30.001075 30402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:30.001340 30402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:30.002486 30402 hybrid_clock.cc:648] HybridClock initialized: now 1786172430002453 us; error 50 us; skew 500 ppm
Aug 08 07:00:29 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172429, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:30.005054 30402 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:30.005501 30402 webserver.cc:460] Webserver started at http://127.24.181.126:33877/ using document root <none> and password file <none>
I20260808 07:00:30.005743 30402 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:30.005808 30402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:30.005980 30402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:30.007184 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "56dae245c6a5494d903d7b67afa600ae"
format_stamp: "Formatted at 2026-08-08 07:00:29 on dist-test-slave-nfb5"
server_key: "34e0dbed2118b081b6ca022fa288a3d5"
server_key_iv: "09a006f7fcdc4b48bc34e8d95ca864ef"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.007560 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "56dae245c6a5494d903d7b67afa600ae"
format_stamp: "Formatted at 2026-08-08 07:00:29 on dist-test-slave-nfb5"
server_key: "34e0dbed2118b081b6ca022fa288a3d5"
server_key_iv: "09a006f7fcdc4b48bc34e8d95ca864ef"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.008955 30402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:30.009694 30431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.009897 30402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:30.010001 30402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "56dae245c6a5494d903d7b67afa600ae"
format_stamp: "Formatted at 2026-08-08 07:00:29 on dist-test-slave-nfb5"
server_key: "34e0dbed2118b081b6ca022fa288a3d5"
server_key_iv: "09a006f7fcdc4b48bc34e8d95ca864ef"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.010092 30402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:30.045239 30402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:30.047279 30402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:30.047458 30402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:30.051625 30402 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:39515
I20260808 07:00:30.051708 30512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:39515 every 8 connection(s)
I20260808 07:00:30.052229 30402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:30.052948 30513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:30.055327 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae: Bootstrap starting.
I20260808 07:00:30.055878 30513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:30.056121 30513 log.cc:824] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:30.056828 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae: No bootstrap required, opened a new log
I20260808 07:00:30.058830 30513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } }
I20260808 07:00:30.059041 30513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:30.059072 30513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dae245c6a5494d903d7b67afa600ae, State: Initialized, Role: FOLLOWER
I20260808 07:00:30.059197 30513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } }
I20260808 07:00:30.059281 30513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:30.059304 30513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:30.059340 30513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:30.060189 30513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } }
I20260808 07:00:30.060278 30513 leader_election.cc:302] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56dae245c6a5494d903d7b67afa600ae; no voters:
I20260808 07:00:30.060420 30513 leader_election.cc:288] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:30.060503 30517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:30.060695 30517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [term 1 LEADER]: Becoming Leader. State: Replica: 56dae245c6a5494d903d7b67afa600ae, State: Running, Role: LEADER
I20260808 07:00:30.060706 30513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:30.060863 30517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } }
I20260808 07:00:30.061229 30519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dae245c6a5494d903d7b67afa600ae. Latest consensus state: current_term: 1 leader_uuid: "56dae245c6a5494d903d7b67afa600ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } } }
I20260808 07:00:30.061229 30518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56dae245c6a5494d903d7b67afa600ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dae245c6a5494d903d7b67afa600ae" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39515 } } }
I20260808 07:00:30.061412 30519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:30.061419 30518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:30.061851 30532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:30.062391 30532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:30.062463 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30402
I20260808 07:00:30.062587 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:30.062798 25301 external_mini_cluster.cc:1428] Setting key 1ecaf1c70b329aab9ce0280588a289ff
I20260808 07:00:30.063939 30532 catalog_manager.cc:1347] Generated new cluster ID: 11e46bc7ce0248bab693e5ea9fc138b9
I20260808 07:00:30.063994 30532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:30.075801 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.063394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.24.181.126:39515)
0808 07:00:30.063658 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.063667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.063700 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:00:30.063812 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.063819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.063872 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.064016 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.064951 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.066310 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.067422 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.068035 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.068134 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.071153 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.071157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.071157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.071170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.074054 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.074452 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.074456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.074456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.074573 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.074794 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.074795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.074796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.075141 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.075297 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.075410 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.075555 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":85,"threads_started":1}
I20260808 07:00:30.077068 30532 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:30.077795 30532 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:30.084049 30532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56dae245c6a5494d903d7b67afa600ae: Generated new TSK 0
I20260808 07:00:30.084218 30532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:30.117448 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:30.219542 30557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:30.219771 30557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:30.219812 30557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:30.221091 30557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:30.221164 30557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:30.221215 30557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:30.222796 30557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:30.223129 30557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:30.223388 30557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:30.228912 30565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.228935 30564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.228914 30568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:30.228977 30557 server_base.cc:1034] running on GCE node
I20260808 07:00:30.229331 30557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:30.229552 30557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:30.230719 30557 hybrid_clock.cc:648] HybridClock initialized: now 1786172430230685 us; error 39 us; skew 500 ppm
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:30.233434 30557 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:30.233889 30557 webserver.cc:460] Webserver started at http://127.24.181.65:39241/ using document root <none> and password file <none>
I20260808 07:00:30.234118 30557 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:30.234167 30557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:30.234302 30557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:30.235476 30557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "d6fb87b3939646cd93152fde5bba78c7"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "61bb67143eabf7c3fa5355ec2aec382b"
server_key_iv: "55feef30ee1cb4d3b9f019b647dde9b4"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.235846 30557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "d6fb87b3939646cd93152fde5bba78c7"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "61bb67143eabf7c3fa5355ec2aec382b"
server_key_iv: "55feef30ee1cb4d3b9f019b647dde9b4"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.237217 30557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:30.238070 30580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.238257 30557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:30.238361 30557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "d6fb87b3939646cd93152fde5bba78c7"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "61bb67143eabf7c3fa5355ec2aec382b"
server_key_iv: "55feef30ee1cb4d3b9f019b647dde9b4"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.238443 30557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:30.251286 30557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:30.253583 30557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:30.253763 30557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:30.254024 30557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:30.254341 30557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:30.254400 30557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.254463 30557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:30.254494 30557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.260401 30557 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:35177
I20260808 07:00:30.260499 30780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:35177 every 8 connection(s)
I20260808 07:00:30.260830 30557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:30.263411 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30557
I20260808 07:00:30.263525 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:30.263725 25301 external_mini_cluster.cc:1428] Setting key 4b914d3e1481dde9d0797fc600c61201
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:30.271308 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.261619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:48457 (local address 127.24.181.126:39515)
0808 07:00:30.261685 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.261687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.261991 (+ 304us) server_negotiation.cc:407] Connection header received
0808 07:00:30.263077 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.263079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.263093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.263139 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.263883 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.264540 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.265592 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.265828 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.265918 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.268325 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.268326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.268327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.268337 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.269847 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.270257 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.270258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.270258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.270327 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.270581 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.270582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.270583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.270706 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.270782 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.271119 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.271218 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:00:30.271559 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.261611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:39515 (local address 127.24.181.65:48457)
0808 07:00:30.261869 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:30.261906 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:30.262954 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:30.263225 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:30.263229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:30.263487 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:30.263772 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.263777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.264630 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.264631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.265494 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.265496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.265931 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.265931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.266057 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.266592 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:30.266600 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:30.268222 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:30.269938 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.269945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.269948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.270177 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.270397 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.270398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.270399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.270511 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.270784 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:30.270792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:30.270972 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 07:00:30.271093 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 07:00:30.271246 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":182,"threads_started":1}
I20260808 07:00:30.272099 30782 heartbeater.cc:347] Connected to a master server at 127.24.181.126:39515
I20260808 07:00:30.272207 30782 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:30.272387 30782 heartbeater.cc:511] Master 127.24.181.126:39515 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:00:30.272925 30455 ts_manager.cc:194] Registered new tserver with Master: d6fb87b3939646cd93152fde5bba78c7 (127.24.181.65:35177)
I20260808 07:00:30.273546 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:48457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:30.290723 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:30.373082 30797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:30.373330 30797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:30.373370 30797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:30.374497 30797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:30.374564 30797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:30.374608 30797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:30.376132 30797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:30.376399 30797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:30.376626 30797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:30.381660 30804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.381644 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.381652 30803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:30.381745 30797 server_base.cc:1034] running on GCE node
I20260808 07:00:30.382066 30797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:30.382265 30797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:30.383431 30797 hybrid_clock.cc:648] HybridClock initialized: now 1786172430383422 us; error 41 us; skew 500 ppm
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:30.385919 30797 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:30.386409 30797 webserver.cc:460] Webserver started at http://127.24.181.66:40981/ using document root <none> and password file <none>
I20260808 07:00:30.386634 30797 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:30.386703 30797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:30.386865 30797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:30.387991 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "6c9bcd07b0cc4b81bede1b62c5206488"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "ca63179d641a8338cec4a7466538bf2d"
server_key_iv: "43fac2c9a939c65fd122674560a92def"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.388360 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "6c9bcd07b0cc4b81bede1b62c5206488"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "ca63179d641a8338cec4a7466538bf2d"
server_key_iv: "43fac2c9a939c65fd122674560a92def"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.389679 30797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:00:30.390401 30819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.390587 30797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:30.390692 30797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "6c9bcd07b0cc4b81bede1b62c5206488"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "ca63179d641a8338cec4a7466538bf2d"
server_key_iv: "43fac2c9a939c65fd122674560a92def"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.390785 30797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:30.399772 30797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:30.401551 30797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:30.401717 30797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:30.401966 30797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:30.402379 30797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:30.402449 30797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.402532 30797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:30.402562 30797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.408676 30797 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:39365
I20260808 07:00:30.408733 30989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:39365 every 8 connection(s)
I20260808 07:00:30.409070 30797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:30.415496 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30797
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:30.415624 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:30.415820 25301 external_mini_cluster.cc:1428] Setting key e0493db74e30a912e4ee8d6c4f129507
I20260808 07:00:30.419262 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.409787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:46895 (local address 127.24.181.126:39515)
0808 07:00:30.409884 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.409887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.410076 (+ 189us) server_negotiation.cc:407] Connection header received
0808 07:00:30.411066 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.411068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.411082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.411141 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.411770 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.412348 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.413423 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.413635 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.413721 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.416160 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.416162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.416163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.416171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.417637 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.418155 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.418157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.418157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.418243 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.418555 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.418556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.418556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.418663 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.418720 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.419079 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.419152 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:00:30.419495 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.409763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:39515 (local address 127.24.181.66:46895)
0808 07:00:30.409938 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:30.409986 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:30.410935 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:30.411211 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:30.411216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:30.411480 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:30.411663 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.411667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.412444 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.412445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.413324 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.413326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.413732 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.413733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.413874 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.414413 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:30.414420 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:30.416048 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:30.417746 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.417754 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.417757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.418054 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.418324 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.418324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.418333 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.418485 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.418731 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:30.418737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:30.418905 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 07:00:30.419057 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 07:00:30.419249 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260808 07:00:30.420077 30990 heartbeater.cc:347] Connected to a master server at 127.24.181.126:39515
I20260808 07:00:30.420186 30990 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:30.420384 30990 heartbeater.cc:511] Master 127.24.181.126:39515 requested a full tablet report, sending...
I20260808 07:00:30.420806 30455 ts_manager.cc:194] Registered new tserver with Master: 6c9bcd07b0cc4b81bede1b62c5206488 (127.24.181.66:39365)
I20260808 07:00:30.421193 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:46895
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:30.444046 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:30.526961 30999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:30.527163 30999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:30.527187 30999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:30.528358 30999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:30.528403 30999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:30.528430 30999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:30.529937 30999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:39515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:30.530175 30999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:30.530416 30999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:30.535462 31008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.535449 31005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:30.535451 31006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:30.535861 30999 server_base.cc:1034] running on GCE node
I20260808 07:00:30.536108 30999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:30.536361 30999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:30.537532 30999 hybrid_clock.cc:648] HybridClock initialized: now 1786172430537524 us; error 38 us; skew 500 ppm
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:30.540153 30999 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:30.540740 30999 webserver.cc:460] Webserver started at http://127.24.181.67:33957/ using document root <none> and password file <none>
I20260808 07:00:30.541002 30999 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:30.541074 30999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:30.541232 30999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:30.542440 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "de8992f26c4d489785b8f9652f00c904"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "d25eef5684345022a724e1ff7a8d14ad"
server_key_iv: "73ea4650510afe4e174462b083c3b415"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.542806 30999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "de8992f26c4d489785b8f9652f00c904"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "d25eef5684345022a724e1ff7a8d14ad"
server_key_iv: "73ea4650510afe4e174462b083c3b415"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.544235 30999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:30.545030 31017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.545262 30999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:30.545374 30999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "de8992f26c4d489785b8f9652f00c904"
format_stamp: "Formatted at 2026-08-08 07:00:30 on dist-test-slave-nfb5"
server_key: "d25eef5684345022a724e1ff7a8d14ad"
server_key_iv: "73ea4650510afe4e174462b083c3b415"
server_key_version: "encryptionkey@0"
I20260808 07:00:30.545449 30999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:30.555598 30999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:30.557796 30999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:30.557969 30999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:30.558210 30999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:30.558539 30999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:30.558597 30999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.558645 30999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:30.558674 30999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:30.565255 30999 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41539
I20260808 07:00:30.565340 31177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41539 every 8 connection(s)
I20260808 07:00:30.565677 30999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:30.569176 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 30999
I20260808 07:00:30.569274 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:30.569478 25301 external_mini_cluster.cc:1428] Setting key f874c57cae1e7a088d0ecbd550a73e87
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:30.575721 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.566494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:34503 (local address 127.24.181.126:39515)
0808 07:00:30.566540 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.566542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.566734 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:00:30.567807 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.567808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.567823 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.567889 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.568546 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.569059 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.570575 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.570797 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.570876 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.573041 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.573042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.573043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.573050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.574202 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.574637 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.574638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.574639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.574705 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.575011 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.575012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.575013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.575140 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.575219 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.575513 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.575600 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:00:30.575901 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.566452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:39515 (local address 127.24.181.67:34503)
0808 07:00:30.566602 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:30.566646 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:30.567666 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:30.567963 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:30.567967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:30.568245 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:30.568442 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.568446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.569177 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.569178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.570472 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:30.570474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.570886 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.570887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:30.571066 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.571532 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:30.571541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:30.572939 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:30.574315 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.574324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.574326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.574556 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.574778 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:30.574779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:30.574779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:30.574895 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:30.575222 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:30.575230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:30.575371 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 07:00:30.575474 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:00:30.575639 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260808 07:00:30.576452 31178 heartbeater.cc:347] Connected to a master server at 127.24.181.126:39515
I20260808 07:00:30.576582 31178 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:30.576778 31178 heartbeater.cc:511] Master 127.24.181.126:39515 requested a full tablet report, sending...
I20260808 07:00:30.577198 30455 ts_manager.cc:194] Registered new tserver with Master: de8992f26c4d489785b8f9652f00c904 (127.24.181.67:41539)
I20260808 07:00:30.577550 30455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:34503
I20260808 07:00:30.581421 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:30.605623 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.597951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.24.181.65:35177)
0808 07:00:30.598150 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.598155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.598165 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:30.598193 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.598194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.598208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.598278 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.598826 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.599592 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.600315 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.600628 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.600693 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.602981 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.602989 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.602990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.602996 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.604517 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.604850 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.604853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.604853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.604914 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.605117 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.605119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.605119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.605292 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.605410 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.605471 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.605538 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":110,"threads_started":1}
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[30381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:30.613796 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:30.607092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.24.181.126:39515)
0808 07:00:30.607162 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:30.607164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:30.607174 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:30.607242 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:30.607244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:30.607255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:30.607331 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:30.607824 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.608339 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.609015 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:30.609229 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:30.609306 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:30.611469 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:30.611471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:30.611471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:30.611477 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:30.612695 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.613048 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.613048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.613048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.613107 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:30.613313 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:30.613314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:30.613315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:30.613471 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:30.613549 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:30.613670 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:00:30.613733 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:00:30.614078 30455 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36696
I20260808 07:00:30.614439 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30557
I20260808 07:00:30.628758 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30797
I20260808 07:00:30.647183 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30999
I20260808 07:00:30.657907 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 30402
2026-08-08T07:00:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2099 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[31194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[31194](info): set up 2 sockets
Aug 08 07:00:30 dist-test-slave-nfb5 krb5kdc[31194](info): commencing operation
krb5kdc: starting...
W20260808 07:00:31.868420 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:31 dist-test-slave-nfb5 krb5kdc[31194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:31.990471 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37785
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:37785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:32.071719 31219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:32.071913 31219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:32.071935 31219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:32.073024 31219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:32.073057 31219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:32.073071 31219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:32.073084 31219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:32.073096 31219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:32.074539 31219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:37785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:32.074714 31219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:32.074880 31219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:32.079669 31229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.079669 31232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:32.079715 31219 server_base.cc:1034] running on GCE node
W20260808 07:00:32.079717 31230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:32.080133 31219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:32.080439 31219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:32.081609 31219 hybrid_clock.cc:648] HybridClock initialized: now 1786172432081594 us; error 28 us; skew 500 ppm
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:32.084247 31219 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:32.084561 31219 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:32.084606 31219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:32.084758 31219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:32.085783 31219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "70d8de8951aa44e798e6b0119de960e4"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "a1a69601ec288e352784d0a742dd8cfb"
server_key_iv: "6f4460d5271ddeaa4facccd389257e2d"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.086091 31219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "70d8de8951aa44e798e6b0119de960e4"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "a1a69601ec288e352784d0a742dd8cfb"
server_key_iv: "6f4460d5271ddeaa4facccd389257e2d"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.087411 31219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:32.088119 31241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.088302 31219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:32.088392 31219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "70d8de8951aa44e798e6b0119de960e4"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "a1a69601ec288e352784d0a742dd8cfb"
server_key_iv: "6f4460d5271ddeaa4facccd389257e2d"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.088449 31219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:32.116631 31219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:32.118485 31219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:32.118602 31219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:32.122637 31219 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:37785
I20260808 07:00:32.122666 31319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:37785 every 8 connection(s)
I20260808 07:00:32.123029 31219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:32.124190 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:32.125351 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 31219
I20260808 07:00:32.125480 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:32.125692 25301 external_mini_cluster.cc:1428] Setting key 8b8cbc2bc602a41f0daefa8d68f7a6d1
I20260808 07:00:32.127218 31320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: Bootstrap starting.
I20260808 07:00:32.127844 31320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:32.128165 31320 log.cc:824] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:32.128890 31320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: No bootstrap required, opened a new log
I20260808 07:00:32.130895 31320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } }
I20260808 07:00:32.131107 31320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:32.131161 31320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d8de8951aa44e798e6b0119de960e4, State: Initialized, Role: FOLLOWER
I20260808 07:00:32.131317 31320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } }
I20260808 07:00:32.131414 31320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:32.131460 31320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:32.131508 31320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:32.132239 31320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } }
I20260808 07:00:32.132355 31320 leader_election.cc:302] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70d8de8951aa44e798e6b0119de960e4; no voters:
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:32.132534 31320 leader_election.cc:288] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:32.132560 31326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:32.132681 31326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [term 1 LEADER]: Becoming Leader. State: Replica: 70d8de8951aa44e798e6b0119de960e4, State: Running, Role: LEADER
I20260808 07:00:32.132828 31320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:32.132839 31326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } }
I20260808 07:00:32.133289 31328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70d8de8951aa44e798e6b0119de960e4. Latest consensus state: current_term: 1 leader_uuid: "70d8de8951aa44e798e6b0119de960e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } } }
I20260808 07:00:32.133459 31328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:32.133683 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70d8de8951aa44e798e6b0119de960e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d8de8951aa44e798e6b0119de960e4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37785 } } }
I20260808 07:00:32.133809 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:32.134330 31337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:32.135067 31337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:32.136786 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.126297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.24.181.126:37785)
0808 07:00:32.126507 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.126515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.126541 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:00:32.127039 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.127047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.127096 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.127258 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:32.127945 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.128852 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.129629 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.130046 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.130170 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.132987 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:32.132990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:32.132991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:32.133003 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:32.135158 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.135510 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.135513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.135513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.135614 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.135834 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.135835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.135836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.136120 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:32.136278 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.136443 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.136547 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260808 07:00:32.139289 31337 catalog_manager.cc:1347] Generated new cluster ID: d8f8211f6f4a4ff3b377d4d46546e1b2
I20260808 07:00:32.139369 31337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:32.157964 31337 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:32.158752 31337 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:32.163231 31337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: Generated new TSK 0
I20260808 07:00:32.163409 31337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:32.193672 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:32.277004 31354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:32.277242 31354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:32.277283 31354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:32.278481 31354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:32.278527 31354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:32.278573 31354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:32.280125 31354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:32.280408 31354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:32.280630 31354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:32.285707 31360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.285823 31366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.285727 31363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:32.285822 31354 server_base.cc:1034] running on GCE node
I20260808 07:00:32.286191 31354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:32.286449 31354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:32.287614 31354 hybrid_clock.cc:648] HybridClock initialized: now 1786172432287603 us; error 58 us; skew 500 ppm
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:32.290171 31354 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:32.290565 31354 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:32.290638 31354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:32.290786 31354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:32.291903 31354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "47dcc1d73dd5407d9fe9e92bb74acd60"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "1acf99b13f986d025071cc8a9c731280"
server_key_iv: "aa183810415f88aa232d16fdef37eb43"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.292333 31354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "47dcc1d73dd5407d9fe9e92bb74acd60"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "1acf99b13f986d025071cc8a9c731280"
server_key_iv: "aa183810415f88aa232d16fdef37eb43"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.293655 31354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:32.294368 31376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.294534 31354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:32.294633 31354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "47dcc1d73dd5407d9fe9e92bb74acd60"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "1acf99b13f986d025071cc8a9c731280"
server_key_iv: "aa183810415f88aa232d16fdef37eb43"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.294734 31354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:32.303434 31354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:32.305702 31354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:32.305868 31354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:32.306103 31354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:32.306420 31354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:32.306476 31354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.306511 31354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:32.306540 31354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.312954 31354 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:41755
I20260808 07:00:32.313009 31529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:41755 every 8 connection(s)
I20260808 07:00:32.313265 31354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:32.319115 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 31354
I20260808 07:00:32.319213 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:32.319401 25301 external_mini_cluster.cc:1428] Setting key 30e5b39b15b247287a5be6a0b65938aa
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:32.323161 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.313974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38175 (local address 127.24.181.126:37785)
0808 07:00:32.314018 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.314020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.314283 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:00:32.314829 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.314830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.314843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.314892 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:32.315923 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.316436 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.317425 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.317636 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.317716 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.319997 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:32.319999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:32.320000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:32.320007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:32.321638 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.322080 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.322081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.322082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.322150 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.322434 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.322435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.322436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.322567 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:32.322657 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.322978 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.323069 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:32.323350 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.313959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37785 (local address 127.24.181.65:38175)
0808 07:00:32.314164 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.314197 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.314713 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.314979 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.314984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.315242 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:32.315812 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.315823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.316537 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.316538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.317332 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.317334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.317724 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.317726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.317876 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.318366 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:32.318374 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:32.319930 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:32.321737 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.321744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.321746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.321996 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.322244 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.322245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.322245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.322352 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.322642 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:32.322650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:32.322819 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.322954 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.323114 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
I20260808 07:00:32.323843 31530 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37785
I20260808 07:00:32.323935 31530 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:32.324098 31530 heartbeater.cc:511] Master 127.24.181.126:37785 requested a full tablet report, sending...
I20260808 07:00:32.324596 31262 ts_manager.cc:194] Registered new tserver with Master: 47dcc1d73dd5407d9fe9e92bb74acd60 (127.24.181.65:41755)
I20260808 07:00:32.325141 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:38175
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:32.346793 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:32.430301 31537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:32.430510 31537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:32.430536 31537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:32.431748 31537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:32.431795 31537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:32.431823 31537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:32.433367 31537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:32.433561 31537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:32.433846 31537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:32.438860 31547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.438894 31545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.439009 31544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:32.439136 31537 server_base.cc:1034] running on GCE node
I20260808 07:00:32.439417 31537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:32.439718 31537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:32.440898 31537 hybrid_clock.cc:648] HybridClock initialized: now 1786172432440877 us; error 40 us; skew 500 ppm
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:32.443454 31537 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:32.443871 31537 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:32.443961 31537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:32.444119 31537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:32.445190 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "6c9a3c3f524743db91070ece2548d9ed"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "0a5cd46d43fd8f4c7ccdeeb68a31ba3e"
server_key_iv: "20541078ced7441cd607568c2e0425d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.445576 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "6c9a3c3f524743db91070ece2548d9ed"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "0a5cd46d43fd8f4c7ccdeeb68a31ba3e"
server_key_iv: "20541078ced7441cd607568c2e0425d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.446894 31537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:32.447682 31557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.447890 31537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:32.447999 31537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "6c9a3c3f524743db91070ece2548d9ed"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "0a5cd46d43fd8f4c7ccdeeb68a31ba3e"
server_key_iv: "20541078ced7441cd607568c2e0425d4"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.448074 31537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:32.476570 31537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:32.478850 31537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:32.479067 31537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:32.479355 31537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:32.479697 31537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:32.479755 31537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.479802 31537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:32.479844 31537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.486079 31537 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:40739
I20260808 07:00:32.486126 31736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:40739 every 8 connection(s)
I20260808 07:00:32.486409 31537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:32.492441 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 31537
I20260808 07:00:32.492583 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:32.492832 25301 external_mini_cluster.cc:1428] Setting key 2076fe4769d7a56656e7c49ca01b9014
I20260808 07:00:32.496523 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.487188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:39067 (local address 127.24.181.126:37785)
0808 07:00:32.487266 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.487268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.487492 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:00:32.487985 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.487986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.487997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.488053 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:32.489016 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.489514 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.490553 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.490752 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.490821 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.493240 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:32.493241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:32.493241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:32.493249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:32.494902 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.495324 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.495325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.495325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.495391 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.495832 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.495833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.495833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.495947 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:32.496004 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.496378 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.496458 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:00:32.496780 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.487178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37785 (local address 127.24.181.66:39067)
0808 07:00:32.487380 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.487418 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.487898 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.488124 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.488127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.488406 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:32.488920 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.488925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.489602 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.489603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.490474 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.490477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.490834 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.490835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.490989 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.491519 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:32.491530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:32.493138 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:32.495024 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.495031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.495033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.495250 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.495573 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.495574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.495574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.495753 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.496019 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:32.496026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:32.496201 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.496330 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.496518 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260808 07:00:32.497360 31737 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37785
I20260808 07:00:32.497460 31737 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:32.497634 31737 heartbeater.cc:511] Master 127.24.181.126:37785 requested a full tablet report, sending...
I20260808 07:00:32.498042 31262 ts_manager.cc:194] Registered new tserver with Master: 6c9a3c3f524743db91070ece2548d9ed (127.24.181.66:40739)
I20260808 07:00:32.498528 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:39067
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:32.521718 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:32.604288 31743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:32.604485 31743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:32.604507 31743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:32.605595 31743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:32.605629 31743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:32.605662 31743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:32.607079 31743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:37785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:32.607271 31743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:32.607426 31743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:32.612174 31750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.612260 31755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:32.612210 31749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:32.612784 31743 server_base.cc:1034] running on GCE node
I20260808 07:00:32.613005 31743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:32.613240 31743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:32.614403 31743 hybrid_clock.cc:648] HybridClock initialized: now 1786172432614395 us; error 36 us; skew 500 ppm
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:32.616983 31743 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:32.617364 31743 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:32.617437 31743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:32.617594 31743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:32.618625 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "726126355837441fbfaf19217754a090"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "dd7ee5a6371112d31c95cf07fc1f8322"
server_key_iv: "8eaa54f6906676a72a7082f6f3c65d03"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.619065 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "726126355837441fbfaf19217754a090"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "dd7ee5a6371112d31c95cf07fc1f8322"
server_key_iv: "8eaa54f6906676a72a7082f6f3c65d03"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.620498 31743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:32.621282 31767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.621474 31743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.621575 31743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "726126355837441fbfaf19217754a090"
format_stamp: "Formatted at 2026-08-08 07:00:32 on dist-test-slave-nfb5"
server_key: "dd7ee5a6371112d31c95cf07fc1f8322"
server_key_iv: "8eaa54f6906676a72a7082f6f3c65d03"
server_key_version: "encryptionkey@0"
I20260808 07:00:32.621703 31743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:32.642270 31743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:32.644194 31743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:32.644354 31743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:32.644603 31743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:32.644948 31743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:32.645004 31743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.645046 31743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:32.645071 31743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:32.651363 31743 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:35231
I20260808 07:00:32.651461 31922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:35231 every 8 connection(s)
I20260808 07:00:32.651782 31743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:32.657003 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 31743
I20260808 07:00:32.657126 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:32.657308 25301 external_mini_cluster.cc:1428] Setting key f754cf8c1d3b38f936bfe52dd635a908
Aug 08 07:00:32 dist-test-slave-nfb5 krb5kdc[31194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172432, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:32.661289 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.652525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:33307 (local address 127.24.181.126:37785)
0808 07:00:32.652604 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.652607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.652758 (+ 151us) server_negotiation.cc:407] Connection header received
0808 07:00:32.653217 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.653218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.653228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.653281 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:32.654182 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.654893 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.655959 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.656144 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.656209 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.658635 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:32.658636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:32.658637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:32.658643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:32.660175 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.660528 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.660529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.660530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.660580 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.660798 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.660798 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.660798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.660899 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:32.660949 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.661171 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.661228 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:00:32.661511 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.652474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37785 (local address 127.24.181.67:33307)
0808 07:00:32.652635 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.652672 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.653134 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.653341 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.653344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.653595 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:32.654092 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.654097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.655046 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.655047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.655867 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.655870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.656227 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.656228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.656335 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.656863 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:32.656874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:32.658555 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:32.660244 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.660250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.660252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.660463 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.660637 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:32.660646 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:32.660647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:32.660743 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:32.660955 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:32.660960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:32.661069 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.661149 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.661264 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":97,"threads_started":1}
I20260808 07:00:32.662166 31923 heartbeater.cc:347] Connected to a master server at 127.24.181.126:37785
I20260808 07:00:32.662248 31923 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:32.662413 31923 heartbeater.cc:511] Master 127.24.181.126:37785 requested a full tablet report, sending...
I20260808 07:00:32.662824 31262 ts_manager.cc:194] Registered new tserver with Master: 726126355837441fbfaf19217754a090 (127.24.181.67:35231)
I20260808 07:00:32.663239 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:33307
I20260808 07:00:32.669041 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:32.675930 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.670266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.24.181.126:37785)
0808 07:00:32.670332 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.670334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.670408 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:00:32.670459 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.670459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.670467 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.670513 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:32.671161 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.671598 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.672290 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.672463 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.672517 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.673587 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:32.673589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:32.673589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:32.673595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:32.675013 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.675291 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.675292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.675292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.675341 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:32.675481 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:32.675482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:32.675482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:32.675600 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:32.675666 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.675778 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.675841 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:00:32.677404 31262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:32.679000 31262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:32.685869 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.24.181.66:40739)
0808 07:00:32.682337 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.682343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.682352 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:32.682389 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.682391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.682405 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.682476 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:32.682712 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683641 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.684969 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685454 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685541 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.685601 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.685730 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.685769 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":77,"threads_started":1}
I20260808 07:00:32.685861 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41755 (local address 127.0.0.1:41600)
0808 07:00:32.681447 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.681493 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.681569 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.681932 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.681935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.681938 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:32.682007 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.682009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683336 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.683337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.684560 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.684563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685521 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.685615 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.685631 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.685680 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.685740 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":12,"thread_start_us":56,"threads_started":1}
I20260808 07:00:32.685863 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:35231 (local address 127.0.0.1:59462)
0808 07:00:32.682038 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.682049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.682093 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.682409 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.682410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.682412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:32.682450 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.682451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683503 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.683504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.684683 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.684686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685594 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.685644 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.685646 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.685680 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.685752 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":27,"thread_start_us":89,"threads_started":1}
I20260808 07:00:32.686020 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.24.181.65:41755)
0808 07:00:32.681738 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.681744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.681757 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:00:32.681800 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.681802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.681818 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.681902 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:32.682097 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683230 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.684650 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685410 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685805 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.685870 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.685897 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.685945 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260808 07:00:32.686376 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.24.181.67:35231)
0808 07:00:32.682247 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.682251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.682261 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:32.682301 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.682303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.682319 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.682395 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:32.682506 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683381 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.684776 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685504 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685894 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.686029 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.686060 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.686308 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":110,"threads_started":1}
I20260808 07:00:32.686409 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.681732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:40739 (local address 127.0.0.1:55380)
0808 07:00:32.682122 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.682133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.682177 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.682574 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.682575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.682578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:32.682626 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.682627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.683742 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.683743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.684877 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.684879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.685541 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.685542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.685611 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.685633 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.685705 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.685752 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":21,"thread_start_us":66,"threads_started":1}
I20260808 07:00:32.686761 31443 tablet_service.cc:1467] Processing CreateTablet for tablet 3f735143d7874cdb982edbaaba66b596 (DEFAULT_TABLE table=test-table [id=7ae3bfbef77b4c42a449838409d279a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:32.687086 31832 tablet_service.cc:1467] Processing CreateTablet for tablet 3f735143d7874cdb982edbaaba66b596 (DEFAULT_TABLE table=test-table [id=7ae3bfbef77b4c42a449838409d279a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:32.687203 31443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f735143d7874cdb982edbaaba66b596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:32.687166 31629 tablet_service.cc:1467] Processing CreateTablet for tablet 3f735143d7874cdb982edbaaba66b596 (DEFAULT_TABLE table=test-table [id=7ae3bfbef77b4c42a449838409d279a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:32.687404 31832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f735143d7874cdb982edbaaba66b596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:32.687502 31629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f735143d7874cdb982edbaaba66b596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:32.690215 31945 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Bootstrap starting.
I20260808 07:00:32.690716 31947 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Bootstrap starting.
I20260808 07:00:32.691068 31945 tablet_bootstrap.cc:654] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:32.691074 31946 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Bootstrap starting.
I20260808 07:00:32.691437 31947 tablet_bootstrap.cc:654] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:32.691664 31946 tablet_bootstrap.cc:654] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:32.691802 31945 log.cc:824] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:32.692080 31947 log.cc:824] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:32.692344 31946 log.cc:824] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:32.692622 31945 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: No bootstrap required, opened a new log
I20260808 07:00:32.692716 31945 ts_tablet_manager.cc:1397] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 07:00:32.692826 31947 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: No bootstrap required, opened a new log
I20260808 07:00:32.692898 31947 ts_tablet_manager.cc:1397] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:00:32.693056 31946 tablet_bootstrap.cc:492] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: No bootstrap required, opened a new log
I20260808 07:00:32.693123 31946 ts_tablet_manager.cc:1397] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:00:32.694384 31945 raft_consensus.cc:348] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.694548 31945 raft_consensus.cc:374] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:32.694595 31945 raft_consensus.cc:729] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9a3c3f524743db91070ece2548d9ed, State: Initialized, Role: FOLLOWER
I20260808 07:00:32.694708 31945 consensus_queue.cc:260] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.695026 31737 heartbeater.cc:503] Master 127.24.181.126:37785 was elected leader, sending a full tablet report...
I20260808 07:00:32.695051 31945 ts_tablet_manager.cc:1428] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:00:32.695000 31947 raft_consensus.cc:348] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.695147 31947 raft_consensus.cc:374] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:32.695181 31947 raft_consensus.cc:729] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726126355837441fbfaf19217754a090, State: Initialized, Role: FOLLOWER
I20260808 07:00:32.695135 31946 raft_consensus.cc:348] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.695286 31946 raft_consensus.cc:374] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:32.695327 31946 raft_consensus.cc:729] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47dcc1d73dd5407d9fe9e92bb74acd60, State: Initialized, Role: FOLLOWER
I20260808 07:00:32.695307 31947 consensus_queue.cc:260] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.695449 31946 consensus_queue.cc:260] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.695587 31947 ts_tablet_manager.cc:1428] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:32.695603 31923 heartbeater.cc:503] Master 127.24.181.126:37785 was elected leader, sending a full tablet report...
I20260808 07:00:32.695698 31946 ts_tablet_manager.cc:1428] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:32.695855 31530 heartbeater.cc:503] Master 127.24.181.126:37785 was elected leader, sending a full tablet report...
W20260808 07:00:32.737030 31739 tablet.cc:2369] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:32.814026 31533 tablet.cc:2369] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:32.902398 31924 tablet.cc:2369] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:32.915361 31952 raft_consensus.cc:482] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:32.915491 31952 raft_consensus.cc:504] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.915828 31952 leader_election.cc:288] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47dcc1d73dd5407d9fe9e92bb74acd60 (127.24.181.65:41755), 726126355837441fbfaf19217754a090 (127.24.181.67:35231)
I20260808 07:00:32.919754 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.916149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:52285 (local address 127.24.181.65:41755)
0808 07:00:32.916233 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.916235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.916262 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:00:32.916296 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.916297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.916308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.916376 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:32.916563 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.917334 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.918782 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.919374 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.919446 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.919454 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.919642 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.919681 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:00:32.919754 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.916379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:54201 (local address 127.24.181.67:35231)
0808 07:00:32.916459 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.916462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.916536 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:00:32.916585 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.916586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.916595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.916652 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:32.916787 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.917456 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.918736 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.919364 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.919446 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.919454 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.919641 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.919679 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:00:32.919783 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.916267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:35231 (local address 127.24.181.66:54201)
0808 07:00:32.916459 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.916468 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.916528 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.916664 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.916665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.916666 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:32.916719 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.916720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.917532 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.917533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.918641 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.918644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.919461 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.919462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.919549 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.919553 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.919618 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.919673 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":21,"thread_start_us":109,"threads_started":1}
I20260808 07:00:32.920001 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.916062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41755 (local address 127.24.181.66:52285)
0808 07:00:32.916136 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:32.916152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:32.916232 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:32.916384 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:32.916386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:32.916388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:32.916470 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.916473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.917415 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.917415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.918641 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:32.918644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.919461 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.919462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:32.919560 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.919563 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:32.919618 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:00:32.919673 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":10}
I20260808 07:00:32.920202 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f735143d7874cdb982edbaaba66b596" candidate_uuid: "6c9a3c3f524743db91070ece2548d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "726126355837441fbfaf19217754a090" is_pre_election: true
I20260808 07:00:32.920311 31476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f735143d7874cdb982edbaaba66b596" candidate_uuid: "6c9a3c3f524743db91070ece2548d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" is_pre_election: true
I20260808 07:00:32.920377 31863 raft_consensus.cc:2454] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c9a3c3f524743db91070ece2548d9ed in term 0.
I20260808 07:00:32.920459 31476 raft_consensus.cc:2454] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c9a3c3f524743db91070ece2548d9ed in term 0.
I20260808 07:00:32.920607 31560 leader_election.cc:302] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c9a3c3f524743db91070ece2548d9ed, 726126355837441fbfaf19217754a090; no voters:
I20260808 07:00:32.920842 31952 raft_consensus.cc:2781] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:32.920919 31952 raft_consensus.cc:482] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:32.920938 31952 raft_consensus.cc:3037] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:32.921680 31952 raft_consensus.cc:504] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.921787 31952 leader_election.cc:288] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [CANDIDATE]: Term 1 election: Requested vote from peers 47dcc1d73dd5407d9fe9e92bb74acd60 (127.24.181.65:41755), 726126355837441fbfaf19217754a090 (127.24.181.67:35231)
I20260808 07:00:32.921979 31476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f735143d7874cdb982edbaaba66b596" candidate_uuid: "6c9a3c3f524743db91070ece2548d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60"
I20260808 07:00:32.922022 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f735143d7874cdb982edbaaba66b596" candidate_uuid: "6c9a3c3f524743db91070ece2548d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "726126355837441fbfaf19217754a090"
I20260808 07:00:32.922084 31476 raft_consensus.cc:3037] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:32.922101 31863 raft_consensus.cc:3037] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:32.923003 31476 raft_consensus.cc:2454] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c9a3c3f524743db91070ece2548d9ed in term 1.
I20260808 07:00:32.923058 31863 raft_consensus.cc:2454] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c9a3c3f524743db91070ece2548d9ed in term 1.
I20260808 07:00:32.923221 31560 leader_election.cc:302] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c9a3c3f524743db91070ece2548d9ed, 726126355837441fbfaf19217754a090; no voters:
I20260808 07:00:32.923310 31952 raft_consensus.cc:2781] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:32.923511 31952 raft_consensus.cc:686] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 1 LEADER]: Becoming Leader. State: Replica: 6c9a3c3f524743db91070ece2548d9ed, State: Running, Role: LEADER
I20260808 07:00:32.923620 31952 consensus_queue.cc:237] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } }
I20260808 07:00:32.924497 31262 catalog_manager.cc:5495] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c9a3c3f524743db91070ece2548d9ed (127.24.181.66). New cstate: current_term: 1 leader_uuid: "6c9a3c3f524743db91070ece2548d9ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c9a3c3f524743db91070ece2548d9ed" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 40739 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:32.967537 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:32.965045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.24.181.66:40739)
0808 07:00:32.965099 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:32.965101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:32.965110 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:32.965159 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:32.965160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:32.965179 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:32.965256 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:32.965425 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.965901 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.966678 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:32.966875 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:32.966998 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:32.967173 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:32.967283 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:32.967385 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:32.967404 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:00:32.967425 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:00:32.969455 31863 raft_consensus.cc:1261] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 1 FOLLOWER]: Refusing update from remote peer 6c9a3c3f524743db91070ece2548d9ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:32.969455 31476 raft_consensus.cc:1261] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 1 FOLLOWER]: Refusing update from remote peer 6c9a3c3f524743db91070ece2548d9ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:32.969849 31959 consensus_queue.cc:1035] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "47dcc1d73dd5407d9fe9e92bb74acd60" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:32.970140 31952 consensus_queue.cc:1035] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "726126355837441fbfaf19217754a090" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:32.972530 31964 mvcc.cc:205] Tried to move back new op lower bound from 7316162285439938560 to 7316162285255561216. Current Snapshot: MvccSnapshot[applied={T|T < 7316162285439938560}]
I20260808 07:00:32.972918 31963 mvcc.cc:205] Tried to move back new op lower bound from 7316162285439938560 to 7316162285255561216. Current Snapshot: MvccSnapshot[applied={T|T < 7316162285439938560}]
I20260808 07:00:32.975157 31260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:32.975270 31260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:32.976054 31260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70d8de8951aa44e798e6b0119de960e4: Sending DeleteTablet for 3 replicas of tablet 3f735143d7874cdb982edbaaba66b596
I20260808 07:00:32.976377 31443 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f735143d7874cdb982edbaaba66b596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:32 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:41600
I20260808 07:00:32.976461 31629 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f735143d7874cdb982edbaaba66b596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:32 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55380
I20260808 07:00:32.976495 31832 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f735143d7874cdb982edbaaba66b596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:32 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:59462
I20260808 07:00:32.976707 31982 tablet_replica.cc:330] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: stopping tablet replica
I20260808 07:00:32.976715 31980 tablet_replica.cc:330] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60: stopping tablet replica
I20260808 07:00:32.976872 31982 raft_consensus.cc:2229] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:32.976912 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 31354
I20260808 07:00:32.976810 31980 raft_consensus.cc:2229] T 3f735143d7874cdb982edbaaba66b596 P 47dcc1d73dd5407d9fe9e92bb74acd60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:32.993119 31981 tablet_replica.cc:330] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: stopping tablet replica
I20260808 07:00:32.993283 31981 raft_consensus.cc:2229] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:32.993438 31981 raft_consensus.cc:2258] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:32.994001 31981 ts_tablet_manager.cc:1905] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:32.995793 31981 ts_tablet_manager.cc:1918] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:32.995936 31981 log.cc:1196] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/3f735143d7874cdb982edbaaba66b596
I20260808 07:00:32.996243 31981 ts_tablet_manager.cc:1939] T 3f735143d7874cdb982edbaaba66b596 P 6c9a3c3f524743db91070ece2548d9ed: Deleting consensus metadata
I20260808 07:00:32.996708 31243 catalog_manager.cc:4841] TS 6c9a3c3f524743db91070ece2548d9ed (127.24.181.66:40739): tablet 3f735143d7874cdb982edbaaba66b596 (table test-table [id=7ae3bfbef77b4c42a449838409d279a3]) successfully deleted
I20260808 07:00:33.003459 31982 raft_consensus.cc:2258] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:33.004016 31982 ts_tablet_manager.cc:1905] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:00:33.005132 31243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:41755 (error 108)
I20260808 07:00:33.005626 31982 ts_tablet_manager.cc:1918] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:33.005752 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.005587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41755 (local address 127.0.0.1:41604)
0808 07:00:33.005662 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.005717 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:41755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:00:33.005805 31982 log.cc:1196] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/3f735143d7874cdb982edbaaba66b596
W20260808 07:00:33.005874 31243 catalog_manager.cc:4584] TS 47dcc1d73dd5407d9fe9e92bb74acd60 (127.24.181.65:41755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f735143d7874cdb982edbaaba66b596: Network error: Client connection negotiation failed: client connection to 127.24.181.65:41755: connect: Connection refused (error 111)
I20260808 07:00:33.006122 31982 ts_tablet_manager.cc:1939] T 3f735143d7874cdb982edbaaba66b596 P 726126355837441fbfaf19217754a090: Deleting consensus metadata
I20260808 07:00:33.006150 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 31537
I20260808 07:00:33.009486 31243 catalog_manager.cc:4841] TS 726126355837441fbfaf19217754a090 (127.24.181.67:35231): tablet 3f735143d7874cdb982edbaaba66b596 (table test-table [id=7ae3bfbef77b4c42a449838409d279a3]) successfully deleted
I20260808 07:00:33.014154 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 31743
I20260808 07:00:33.022368 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 31219
2026-08-08T07:00:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2362 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T07:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:33Z Disabled control of system clock
I20260808 07:00:33.054842 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44951
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:00:33.135890 31989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:33.136086 31989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:33.136107 31989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:33.137264 31989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:33.137306 31989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:33.137319 31989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:33.137332 31989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:33.137392 31989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:33.138736 31989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:33.139008 31989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:33.139307 31989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:33.144157 32003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.144133 32000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.144176 32001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.144399 31989 server_base.cc:1034] running on GCE node
I20260808 07:00:33.144673 31989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:33.144922 31989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:33.146090 31989 hybrid_clock.cc:648] HybridClock initialized: now 1786172433146079 us; error 33 us; skew 500 ppm
I20260808 07:00:33.147363 31989 webserver.cc:460] Webserver started at http://127.24.181.126:33751/ using document root <none> and password file <none>
I20260808 07:00:33.147619 31989 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:33.147689 31989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:33.147858 31989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:33.148947 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "52c4683ca9c843eaaf62d1392d9c98d4"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "66282a32ceb75d6c1813f78dca4a6d2e"
server_key_iv: "b57ea0b50d3c514379cde1ae1fc10a53"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.149325 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "52c4683ca9c843eaaf62d1392d9c98d4"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "66282a32ceb75d6c1813f78dca4a6d2e"
server_key_iv: "b57ea0b50d3c514379cde1ae1fc10a53"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.150662 31989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:33.151492 32013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.151669 31989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.151778 31989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "52c4683ca9c843eaaf62d1392d9c98d4"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "66282a32ceb75d6c1813f78dca4a6d2e"
server_key_iv: "b57ea0b50d3c514379cde1ae1fc10a53"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.151865 31989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:33.155224 31989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:33.155507 31989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:33.155666 31989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:33.159631 31989 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44951
I20260808 07:00:33.159688 32082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44951 every 8 connection(s)
I20260808 07:00:33.160032 31989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:33.160781 32083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:33.163563 32083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: Bootstrap starting.
I20260808 07:00:33.164072 32083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:33.164325 32083 log.cc:824] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:33.164943 32083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: No bootstrap required, opened a new log
I20260808 07:00:33.166458 32083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } }
I20260808 07:00:33.166590 32083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:33.166613 32083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52c4683ca9c843eaaf62d1392d9c98d4, State: Initialized, Role: FOLLOWER
I20260808 07:00:33.166754 32083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } }
I20260808 07:00:33.166837 32083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:33.166860 32083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:33.166890 32083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:33.167447 32083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } }
I20260808 07:00:33.167534 32083 leader_election.cc:302] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52c4683ca9c843eaaf62d1392d9c98d4; no voters:
I20260808 07:00:33.167661 32083 leader_election.cc:288] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:33.167744 32086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:33.167897 32086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [term 1 LEADER]: Becoming Leader. State: Replica: 52c4683ca9c843eaaf62d1392d9c98d4, State: Running, Role: LEADER
I20260808 07:00:33.167924 32083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:33.167989 32086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } }
I20260808 07:00:33.168386 32088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52c4683ca9c843eaaf62d1392d9c98d4. Latest consensus state: current_term: 1 leader_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } } }
I20260808 07:00:33.168479 32088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:33.168599 32087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c4683ca9c843eaaf62d1392d9c98d4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44951 } } }
I20260808 07:00:33.168672 32087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:33.168711 32101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:33.169019 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 31989
I20260808 07:00:33.169134 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:33.169327 25301 external_mini_cluster.cc:1428] Setting key 4c020018e49d77463239dda7e0604704
I20260808 07:00:33.169359 32101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:33.171561 32101 catalog_manager.cc:1347] Generated new cluster ID: fd14090423584143866adcb697e9d69a
I20260808 07:00:33.171614 32101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:33.172021 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.169989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.24.181.126:44951)
0808 07:00:33.170177 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.170186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.170284 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:00:33.170401 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.170409 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.170453 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.170601 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.171385 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.171388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.171388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.171393 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.171440 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.171455 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.171750 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.171781 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.171847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260808 07:00:33.188359 32101 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:33.188544 32101 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:33.198040 32101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: Generated new TSK 0
I20260808 07:00:33.198237 32101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:33.205544 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--builtin_ntp_servers=127.24.181.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:00:33.292619 32117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:33.292869 32117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:33.292907 32117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:33.294071 32117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:33.294137 32117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:33.294179 32117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:33.296077 32117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:33.296386 32117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:33.296726 32117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:33.302246 32128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.302275 32125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.302389 32117 server_base.cc:1034] running on GCE node
W20260808 07:00:33.302251 32126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.302695 32117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:33.302901 32117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:33.304113 32117 hybrid_clock.cc:648] HybridClock initialized: now 1786172433304100 us; error 32 us; skew 500 ppm
I20260808 07:00:33.305299 32117 webserver.cc:460] Webserver started at http://127.24.181.65:35533/ using document root <none> and password file <none>
I20260808 07:00:33.305562 32117 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:33.305610 32117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:33.305784 32117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:33.306846 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "00a94926413d4fd1b2b694523e19e184"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "933fdaf385d60e3f5a4b4bc00d02ba3c"
server_key_iv: "f2dae469a04f3fd6d3e6f973fc4c5bdb"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.307330 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "00a94926413d4fd1b2b694523e19e184"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "933fdaf385d60e3f5a4b4bc00d02ba3c"
server_key_iv: "f2dae469a04f3fd6d3e6f973fc4c5bdb"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.308616 32117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:00:33.309432 32139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.309612 32117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:33.309706 32117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "00a94926413d4fd1b2b694523e19e184"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "933fdaf385d60e3f5a4b4bc00d02ba3c"
server_key_iv: "f2dae469a04f3fd6d3e6f973fc4c5bdb"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.309799 32117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:33.313123 32117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:33.313385 32117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:33.313529 32117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:33.313743 32117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:33.314088 32117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:33.314145 32117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.314185 32117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:33.314213 32117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.319837 32117 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36951
I20260808 07:00:33.319890 32290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36951 every 8 connection(s)
I20260808 07:00:33.320225 32117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:33.320996 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 32117
I20260808 07:00:33.321118 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:33.321339 25301 external_mini_cluster.cc:1428] Setting key b915f0d9affc2415706161ea27289016
I20260808 07:00:33.322368 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--builtin_ntp_servers=127.24.181.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 07:00:33.323616 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.321033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38169 (local address 127.24.181.126:44951)
0808 07:00:33.321096 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.321098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.321348 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:00:33.321974 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.321974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.321981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.322027 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.323313 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.323315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.323315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.323326 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.323345 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.323349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.323424 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.323523 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.323542 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:00:33.323837 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.321012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44951 (local address 127.24.181.65:38169)
0808 07:00:33.321223 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.321256 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.321884 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.322092 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.322096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.322105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.323148 (+ 1043us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.323148 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.323176 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.323203 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.323423 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.323430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.323433 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.323534 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.323596 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260808 07:00:33.324338 32292 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44951
I20260808 07:00:33.324442 32292 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:33.324591 32292 heartbeater.cc:511] Master 127.24.181.126:44951 requested a full tablet report, sending...
I20260808 07:00:33.325178 32034 ts_manager.cc:194] Registered new tserver with Master: 00a94926413d4fd1b2b694523e19e184 (127.24.181.65:36951)
W20260808 07:00:33.405786 32295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:33.405993 32295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:33.406018 32295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:33.407203 32295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:33.407241 32295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:33.407267 32295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:33.408603 32295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:33.408793 32295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:33.409024 32295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:33.413936 32306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.413967 32311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.413976 32307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.414335 32295 server_base.cc:1034] running on GCE node
I20260808 07:00:33.414583 32295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:33.414829 32295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:33.415987 32295 hybrid_clock.cc:648] HybridClock initialized: now 1786172433415971 us; error 34 us; skew 500 ppm
I20260808 07:00:33.417192 32295 webserver.cc:460] Webserver started at http://127.24.181.66:39379/ using document root <none> and password file <none>
I20260808 07:00:33.417450 32295 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:33.417521 32295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:33.417670 32295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:33.418712 32295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "f07e5d7f61ad4e70904e901020674911"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "6544b81257281b3b9c3cad39aa658cf9"
server_key_iv: "542aeb876855e8cda47070bbacc6eefc"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.419131 32295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "f07e5d7f61ad4e70904e901020674911"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "6544b81257281b3b9c3cad39aa658cf9"
server_key_iv: "542aeb876855e8cda47070bbacc6eefc"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.420580 32295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:33.421334 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.421511 32295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:33.421613 32295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "f07e5d7f61ad4e70904e901020674911"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "6544b81257281b3b9c3cad39aa658cf9"
server_key_iv: "542aeb876855e8cda47070bbacc6eefc"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.421694 32295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:33.425220 32295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:33.425490 32295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:33.425645 32295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:33.425864 32295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:33.426188 32295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:33.426244 32295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.426286 32295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:33.426316 32295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.432385 32295 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:35231
I20260808 07:00:33.432497 32469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:35231 every 8 connection(s)
I20260808 07:00:33.432780 32295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:33.435860 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.433508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43449 (local address 127.24.181.126:44951)
0808 07:00:33.433556 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.433558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.433762 (+ 204us) server_negotiation.cc:407] Connection header received
0808 07:00:33.434298 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.434299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.434306 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.434372 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.435573 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.435574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.435575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.435580 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.435598 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.435603 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.435668 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.435770 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.435787 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:00:33.436062 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.433454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44951 (local address 127.24.181.66:43449)
0808 07:00:33.433638 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.433672 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.434186 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.434439 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.434442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.434452 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.435420 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.435420 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.435447 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.435473 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.435668 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.435674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.435677 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.435779 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.435840 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260808 07:00:33.436595 32470 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44951
I20260808 07:00:33.436689 32470 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:33.436805 32470 heartbeater.cc:511] Master 127.24.181.126:44951 requested a full tablet report, sending...
I20260808 07:00:33.437155 32034 ts_manager.cc:194] Registered new tserver with Master: f07e5d7f61ad4e70904e901020674911 (127.24.181.66:35231)
I20260808 07:00:33.438086 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 32295
I20260808 07:00:33.438184 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:33.438329 25301 external_mini_cluster.cc:1428] Setting key 4f6e92387d023111b6168713804fa6d3
I20260808 07:00:33.439199 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--builtin_ntp_servers=127.24.181.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:00:33.520316 32477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:33.520547 32477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:33.520586 32477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:33.521729 32477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:33.521800 32477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:33.521852 32477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:33.523259 32477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:33.523522 32477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:33.523808 32477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:33.528951 32487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:33.528951 32484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.529174 32477 server_base.cc:1034] running on GCE node
W20260808 07:00:33.529186 32483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:33.529472 32477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:33.529663 32477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:33.530853 32477 hybrid_clock.cc:648] HybridClock initialized: now 1786172433530839 us; error 27 us; skew 500 ppm
I20260808 07:00:33.532066 32477 webserver.cc:460] Webserver started at http://127.24.181.67:43043/ using document root <none> and password file <none>
I20260808 07:00:33.532344 32477 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:33.532415 32477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:33.532572 32477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:33.533628 32477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "197bf87b7ece4998902166b4f00e21ee"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "30b51135a4185c896c723e48f33f50d5"
server_key_iv: "a224e4e43b6c8175a7893d76f86067de"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.534006 32477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "197bf87b7ece4998902166b4f00e21ee"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "30b51135a4185c896c723e48f33f50d5"
server_key_iv: "a224e4e43b6c8175a7893d76f86067de"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.535319 32477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:33.535986 32501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.536167 32477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:33.536249 32477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "197bf87b7ece4998902166b4f00e21ee"
format_stamp: "Formatted at 2026-08-08 07:00:33 on dist-test-slave-nfb5"
server_key: "30b51135a4185c896c723e48f33f50d5"
server_key_iv: "a224e4e43b6c8175a7893d76f86067de"
server_key_version: "encryptionkey@0"
I20260808 07:00:33.536303 32477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:33.539528 32477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:33.539744 32477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:33.539839 32477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:33.539994 32477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:33.540233 32477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:33.540263 32477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.540282 32477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:33.540295 32477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:33.546025 32477 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:45813
I20260808 07:00:33.546103 32662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:45813 every 8 connection(s)
I20260808 07:00:33.546384 32477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:33.549389 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.547123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:45689 (local address 127.24.181.126:44951)
0808 07:00:33.547167 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.547169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.547405 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:00:33.547907 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.547909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.547916 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.547981 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.549142 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.549143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.549144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.549148 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.549166 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.549171 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.549231 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.549302 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.549319 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:00:33.549598 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.547096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44951 (local address 127.24.181.67:45689)
0808 07:00:33.547266 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.547299 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.547800 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.548050 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.548054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.548064 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.548984 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.548985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.549013 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.549040 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.549233 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.549240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.549243 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.549314 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.549385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260808 07:00:33.550137 32663 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44951
I20260808 07:00:33.550227 32663 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:33.550400 32663 heartbeater.cc:511] Master 127.24.181.126:44951 requested a full tablet report, sending...
I20260808 07:00:33.550837 32034 ts_manager.cc:194] Registered new tserver with Master: 197bf87b7ece4998902166b4f00e21ee (127.24.181.67:45813)
I20260808 07:00:33.554229 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 32477
I20260808 07:00:33.554338 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:33.554538 25301 external_mini_cluster.cc:1428] Setting key 1a9f3b1f8e3276a346581462d9157aff
I20260808 07:00:33.555428 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:33.557766 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.556721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.24.181.126:44951)
0808 07:00:33.556793 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.556795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.556868 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:00:33.556919 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.556919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.556924 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.556960 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.557570 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.557570 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.557571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.557573 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.557585 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.557588 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.557626 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.557659 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.557670 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:00:33.559334 32034 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:33.560818 32034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:33.564812 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:35231 (local address 127.0.0.1:54278)
0808 07:00:33.563329 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.563371 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.563444 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.563805 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.563812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.563819 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.564329 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.564330 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.564347 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.564368 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.564629 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.564637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.564642 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.564739 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.564763 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":210,"threads_started":1}
I20260808 07:00:33.565155 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.24.181.66:35231)
0808 07:00:33.563331 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.563336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.563650 (+ 314us) server_negotiation.cc:407] Connection header received
0808 07:00:33.563683 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.563686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.563698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.563785 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.564463 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.564472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.564473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.564478 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.564516 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.564545 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.564602 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.565043 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.565061 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260808 07:00:33.565483 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36951 (local address 127.0.0.1:59422)
0808 07:00:33.564114 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.564122 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.564194 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.564363 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.564364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.564367 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.565085 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.565085 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.565093 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.565106 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.565357 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.565358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.565358 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.565403 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.565417 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":89,"threads_started":1}
I20260808 07:00:33.565732 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:45813 (local address 127.0.0.1:51536)
0808 07:00:33.564287 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.564297 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.564337 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.564744 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.564746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.564748 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.565148 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.565148 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.565157 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.565171 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.565624 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.565624 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.565625 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.565666 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.565678 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":92,"threads_started":1}
I20260808 07:00:33.565738 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.24.181.65:36951)
0808 07:00:33.563912 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.563917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.564194 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:00:33.564248 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.564252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.564263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.564345 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.565180 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.565188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.565189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.565193 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.565228 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.565257 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.565328 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.565634 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.565652 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":95,"threads_started":1}
I20260808 07:00:33.565994 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.563670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.24.181.67:45813)
0808 07:00:33.564085 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.564090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.564395 (+ 305us) server_negotiation.cc:407] Connection header received
0808 07:00:33.564424 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.564426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.564436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.564504 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.565460 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.565468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.565468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.565472 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.565505 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.565529 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.565573 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.565890 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.565907 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":106,"threads_started":1}
I20260808 07:00:33.565943 32386 tablet_service.cc:1467] Processing CreateTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 (DEFAULT_TABLE table=test-table [id=6334d5b0d668485796d26a329427f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:33.566321 32386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e924fd08154bbbbaca152a818f5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:33.566330 32207 tablet_service.cc:1467] Processing CreateTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 (DEFAULT_TABLE table=test-table [id=6334d5b0d668485796d26a329427f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:33.566675 32207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e924fd08154bbbbaca152a818f5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:33.566579 32566 tablet_service.cc:1467] Processing CreateTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 (DEFAULT_TABLE table=test-table [id=6334d5b0d668485796d26a329427f03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:33.566892 32566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e924fd08154bbbbaca152a818f5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:33.569412 32680 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Bootstrap starting.
I20260808 07:00:33.569821 32681 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Bootstrap starting.
I20260808 07:00:33.570212 32682 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Bootstrap starting.
I20260808 07:00:33.570451 32680 tablet_bootstrap.cc:654] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:33.570451 32681 tablet_bootstrap.cc:654] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:33.570796 32681 log.cc:824] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:33.570873 32682 tablet_bootstrap.cc:654] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:33.570850 32680 log.cc:824] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:33.571267 32682 log.cc:824] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:33.571628 32681 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: No bootstrap required, opened a new log
I20260808 07:00:33.571708 32681 ts_tablet_manager.cc:1397] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:33.571768 32680 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: No bootstrap required, opened a new log
I20260808 07:00:33.571838 32680 ts_tablet_manager.cc:1397] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:33.571996 32682 tablet_bootstrap.cc:492] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: No bootstrap required, opened a new log
I20260808 07:00:33.572080 32682 ts_tablet_manager.cc:1397] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:33.574066 32682 raft_consensus.cc:348] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574226 32682 raft_consensus.cc:374] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:33.574271 32682 raft_consensus.cc:729] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197bf87b7ece4998902166b4f00e21ee, State: Initialized, Role: FOLLOWER
I20260808 07:00:33.574385 32682 consensus_queue.cc:260] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574414 32680 raft_consensus.cc:348] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574445 32681 raft_consensus.cc:348] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574576 32680 raft_consensus.cc:374] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:33.574576 32681 raft_consensus.cc:374] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:33.574617 32680 raft_consensus.cc:729] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a94926413d4fd1b2b694523e19e184, State: Initialized, Role: FOLLOWER
I20260808 07:00:33.574617 32681 raft_consensus.cc:729] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07e5d7f61ad4e70904e901020674911, State: Initialized, Role: FOLLOWER
I20260808 07:00:33.574662 32682 ts_tablet_manager.cc:1428] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:00:33.574729 32680 consensus_queue.cc:260] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574729 32681 consensus_queue.cc:260] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.574796 32663 heartbeater.cc:503] Master 127.24.181.126:44951 was elected leader, sending a full tablet report...
I20260808 07:00:33.575011 32680 ts_tablet_manager.cc:1428] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:00:33.575052 32292 heartbeater.cc:503] Master 127.24.181.126:44951 was elected leader, sending a full tablet report...
I20260808 07:00:33.575011 32681 ts_tablet_manager.cc:1428] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:00:33.575238 32470 heartbeater.cc:503] Master 127.24.181.126:44951 was elected leader, sending a full tablet report...
W20260808 07:00:33.683346 32471 tablet.cc:2369] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:33.796931 32665 tablet.cc:2369] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:33.820928 32293 tablet.cc:2369] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:33.923564 32689 raft_consensus.cc:482] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:33.923691 32689 raft_consensus.cc:504] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.924103 32689 leader_election.cc:288] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00a94926413d4fd1b2b694523e19e184 (127.24.181.65:36951), 197bf87b7ece4998902166b4f00e21ee (127.24.181.67:45813)
I20260808 07:00:33.925694 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.924425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:56545 (local address 127.24.181.65:36951)
0808 07:00:33.924495 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.924497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.924507 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:33.924536 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.924537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.924543 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.924605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.925314 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.925331 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.925331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.925351 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.925371 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.925377 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.925501 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.925567 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.925580 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:33.925729 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.924312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36951 (local address 127.24.181.66:56545)
0808 07:00:33.924378 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.924390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.924458 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.924615 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.924616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.924619 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.925197 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.925198 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.925206 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.925221 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.925500 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.925501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.925502 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.925558 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.925574 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:00:33.925829 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.924563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60291 (local address 127.24.181.67:45813)
0808 07:00:33.924626 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:33.924628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:33.924636 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:33.924789 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:33.924790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:33.924796 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:33.924863 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:33.925533 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:33.925535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:33.925535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:33.925543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:33.925561 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:33.925567 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:33.925640 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:33.925740 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:00:33.925755 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:00:33.925997 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:33.924463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:45813 (local address 127.24.181.66:60291)
0808 07:00:33.924570 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:33.924579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:33.924616 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:33.924865 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:33.924867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:33.924869 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:33.925395 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:00:33.925396 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:33.925403 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:00:33.925416 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:33.925667 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:33.925668 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:33.925669 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:33.925913 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 07:00:33.925929 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
I20260808 07:00:33.926291 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e924fd08154bbbbaca152a818f5ad3" candidate_uuid: "f07e5d7f61ad4e70904e901020674911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a94926413d4fd1b2b694523e19e184" is_pre_election: true
I20260808 07:00:33.926342 32596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e924fd08154bbbbaca152a818f5ad3" candidate_uuid: "f07e5d7f61ad4e70904e901020674911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197bf87b7ece4998902166b4f00e21ee" is_pre_election: true
I20260808 07:00:33.926470 32238 raft_consensus.cc:2454] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07e5d7f61ad4e70904e901020674911 in term 0.
I20260808 07:00:33.926470 32596 raft_consensus.cc:2454] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07e5d7f61ad4e70904e901020674911 in term 0.
I20260808 07:00:33.926703 32330 leader_election.cc:302] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197bf87b7ece4998902166b4f00e21ee, f07e5d7f61ad4e70904e901020674911; no voters:
I20260808 07:00:33.926833 32689 raft_consensus.cc:2781] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:33.926892 32689 raft_consensus.cc:482] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:33.926941 32689 raft_consensus.cc:3037] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:33.927671 32689 raft_consensus.cc:504] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.927807 32689 leader_election.cc:288] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [CANDIDATE]: Term 1 election: Requested vote from peers 00a94926413d4fd1b2b694523e19e184 (127.24.181.65:36951), 197bf87b7ece4998902166b4f00e21ee (127.24.181.67:45813)
I20260808 07:00:33.927999 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e924fd08154bbbbaca152a818f5ad3" candidate_uuid: "f07e5d7f61ad4e70904e901020674911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a94926413d4fd1b2b694523e19e184"
I20260808 07:00:33.928078 32238 raft_consensus.cc:3037] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:33.928038 32596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e924fd08154bbbbaca152a818f5ad3" candidate_uuid: "f07e5d7f61ad4e70904e901020674911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197bf87b7ece4998902166b4f00e21ee"
I20260808 07:00:33.928143 32596 raft_consensus.cc:3037] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:33.928874 32238 raft_consensus.cc:2454] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07e5d7f61ad4e70904e901020674911 in term 1.
I20260808 07:00:33.928951 32596 raft_consensus.cc:2454] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07e5d7f61ad4e70904e901020674911 in term 1.
I20260808 07:00:33.929107 32330 leader_election.cc:302] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a94926413d4fd1b2b694523e19e184, f07e5d7f61ad4e70904e901020674911; no voters:
I20260808 07:00:33.929278 32689 raft_consensus.cc:2781] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:33.929481 32689 raft_consensus.cc:686] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 1 LEADER]: Becoming Leader. State: Replica: f07e5d7f61ad4e70904e901020674911, State: Running, Role: LEADER
I20260808 07:00:33.929586 32689 consensus_queue.cc:237] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } }
I20260808 07:00:33.930455 32033 catalog_manager.cc:5495] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 reported cstate change: term changed from 0 to 1, leader changed from <none> to f07e5d7f61ad4e70904e901020674911 (127.24.181.66). New cstate: current_term: 1 leader_uuid: "f07e5d7f61ad4e70904e901020674911" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e5d7f61ad4e70904e901020674911" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 } health_report { overall_health: UNKNOWN } } }
I20260808 07:00:34.008236 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.007218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.24.181.66:35231)
0808 07:00:34.007288 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.007290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.007300 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:34.007347 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.007348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.007363 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.007417 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:34.008037 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:34.008038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:34.008039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:34.008043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:34.008060 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:34.008064 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:34.008116 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:34.008159 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:00:34.008172 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:00:34.010300 32238 raft_consensus.cc:1261] T 55e924fd08154bbbbaca152a818f5ad3 P 00a94926413d4fd1b2b694523e19e184 [term 1 FOLLOWER]: Refusing update from remote peer f07e5d7f61ad4e70904e901020674911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:34.010299 32596 raft_consensus.cc:1261] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 1 FOLLOWER]: Refusing update from remote peer f07e5d7f61ad4e70904e901020674911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:34.010686 32689 consensus_queue.cc:1035] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197bf87b7ece4998902166b4f00e21ee" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:34.010854 32694 consensus_queue.cc:1035] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a94926413d4fd1b2b694523e19e184" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:34.016247 32033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:34.016353 32033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:34.017204 32033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52c4683ca9c843eaaf62d1392d9c98d4: Sending DeleteTablet for 3 replicas of tablet 55e924fd08154bbbbaca152a818f5ad3
I20260808 07:00:34.017556 32386 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:34 UTC) from {username='slave'} at 127.0.0.1:54278
I20260808 07:00:34.017634 32207 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:34 UTC) from {username='slave'} at 127.0.0.1:59422
I20260808 07:00:34.017639 32566 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e924fd08154bbbbaca152a818f5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:34 UTC) from {username='slave'} at 127.0.0.1:51536
I20260808 07:00:34.017956 32711 tablet_replica.cc:330] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: stopping tablet replica
I20260808 07:00:34.017998 32713 tablet_replica.cc:330] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: stopping tablet replica
I20260808 07:00:34.018105 32713 raft_consensus.cc:2229] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:34.018129 32711 raft_consensus.cc:2229] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:34.018224 32713 raft_consensus.cc:2258] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:34.018224 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 32117
I20260808 07:00:34.018497 32711 raft_consensus.cc:2258] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:34.018886 32713 ts_tablet_manager.cc:1905] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:34.020714 32713 ts_tablet_manager.cc:1918] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:34.020869 32713 log.cc:1196] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/55e924fd08154bbbbaca152a818f5ad3
I20260808 07:00:34.021198 32711 ts_tablet_manager.cc:1905] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:34.021261 32713 ts_tablet_manager.cc:1939] T 55e924fd08154bbbbaca152a818f5ad3 P 197bf87b7ece4998902166b4f00e21ee: Deleting consensus metadata
I20260808 07:00:34.021737 32017 catalog_manager.cc:4841] TS 197bf87b7ece4998902166b4f00e21ee (127.24.181.67:45813): tablet 55e924fd08154bbbbaca152a818f5ad3 (table test-table [id=6334d5b0d668485796d26a329427f03a]) successfully deleted
I20260808 07:00:34.022832 32711 ts_tablet_manager.cc:1918] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:34.022996 32711 log.cc:1196] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/55e924fd08154bbbbaca152a818f5ad3
I20260808 07:00:34.023303 32711 ts_tablet_manager.cc:1939] T 55e924fd08154bbbbaca152a818f5ad3 P f07e5d7f61ad4e70904e901020674911: Deleting consensus metadata
I20260808 07:00:34.023725 32017 catalog_manager.cc:4841] TS f07e5d7f61ad4e70904e901020674911 (127.24.181.66:35231): tablet 55e924fd08154bbbbaca152a818f5ad3 (table test-table [id=6334d5b0d668485796d26a329427f03a]) successfully deleted
W20260808 07:00:34.031661 32017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:36951 (error 108)
I20260808 07:00:34.031982 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 32295
I20260808 07:00:34.033850 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.033678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36951 (local address 127.0.0.1:59434)
0808 07:00:34.033735 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:34.033792 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:00:34.034796 32017 catalog_manager.cc:4584] TS 00a94926413d4fd1b2b694523e19e184 (127.24.181.65:36951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55e924fd08154bbbbaca152a818f5ad3: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36951: connect: Connection refused (error 111)
I20260808 07:00:34.040132 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 32477
I20260808 07:00:34.050446 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 31989
2026-08-08T07:00:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1028 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 07:00:34.075510 25301 mini_oidc.cc:152] Starting JWKS server
I20260808 07:00:34.076567 25301 webserver.cc:460] Webserver started at https://127.0.0.1:34683/ using document root <none> and password file <none>
I20260808 07:00:34.076642 25301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:00:34.076793 25301 webserver.cc:460] Webserver started at http://127.0.0.1:39963/ using document root <none> and password file <none>
2026-08-08T07:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:34Z Disabled control of system clock
I20260808 07:00:34.085373 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37303
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:37303
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34683/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/testchainca.cert with env {}
W20260808 07:00:34.167285 32725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:34.167483 32725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:34.167506 32725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:34.168608 32725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:34.168643 32725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:34.168656 32725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:34.168669 32725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:34.168681 32725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:00:34.168694 32725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:00:34.168708 32725 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34683/jwks.json
I20260808 07:00:34.170290 32725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:37303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34683/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:34.170482 32725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:34.170645 32725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:34.175485 32732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:34.175498 32733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.175657 32725 server_base.cc:1034] running on GCE node
W20260808 07:00:34.175498 32735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.176012 32725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:34.176296 32725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:34.177484 32725 hybrid_clock.cc:648] HybridClock initialized: now 1786172434177454 us; error 54 us; skew 500 ppm
I20260808 07:00:34.178647 32725 webserver.cc:460] Webserver started at http://127.24.181.126:37305/ using document root <none> and password file <none>
I20260808 07:00:34.178906 32725 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:34.179039 32725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:34.179206 32725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:34.180253 32725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "e5042567308848c2a6bd5c30deee8734"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "9c7044348109b6b5e9f967c70d1ee494"
server_key_iv: "c66e64a5ae6d2cad971bc6fd2d8cbe92"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.180629 32725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "e5042567308848c2a6bd5c30deee8734"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "9c7044348109b6b5e9f967c70d1ee494"
server_key_iv: "c66e64a5ae6d2cad971bc6fd2d8cbe92"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.181941 32725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:34.182655 32744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:34.182825 32725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:34.182963 32725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "e5042567308848c2a6bd5c30deee8734"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "9c7044348109b6b5e9f967c70d1ee494"
server_key_iv: "c66e64a5ae6d2cad971bc6fd2d8cbe92"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.183043 32725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:34.206450 32725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:34.206779 32725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:34.206993 32725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:34.210983 32725 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:37303
I20260808 07:00:34.211035 345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:37303 every 8 connection(s)
I20260808 07:00:34.211401 32725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:34.212424 346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:34.214728 346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: Bootstrap starting.
I20260808 07:00:34.215376 346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:34.215685 346 log.cc:824] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:34.216372 346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: No bootstrap required, opened a new log
I20260808 07:00:34.217936 346 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } }
I20260808 07:00:34.218070 346 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:34.218124 346 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5042567308848c2a6bd5c30deee8734, State: Initialized, Role: FOLLOWER
I20260808 07:00:34.218300 346 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } }
I20260808 07:00:34.218389 346 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:34.218441 346 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:34.218492 346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:34.219017 346 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } }
I20260808 07:00:34.219132 346 leader_election.cc:302] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5042567308848c2a6bd5c30deee8734; no voters:
I20260808 07:00:34.219336 346 leader_election.cc:288] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:34.219385 349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:34.219554 349 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [term 1 LEADER]: Becoming Leader. State: Replica: e5042567308848c2a6bd5c30deee8734, State: Running, Role: LEADER
I20260808 07:00:34.219594 346 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:34.219664 349 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } }
I20260808 07:00:34.219923 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 32725
I20260808 07:00:34.220026 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:34.220257 25301 external_mini_cluster.cc:1428] Setting key b65a6e1eab239c9fc3d34ded2734cebe
I20260808 07:00:34.220893 353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5042567308848c2a6bd5c30deee8734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } } }
I20260808 07:00:34.220959 354 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5042567308848c2a6bd5c30deee8734. Latest consensus state: current_term: 1 leader_uuid: "e5042567308848c2a6bd5c30deee8734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5042567308848c2a6bd5c30deee8734" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37303 } } }
I20260808 07:00:34.221102 354 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.221179 353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734 [sys.catalog]: This master's current role is: LEADER
W20260808 07:00:34.221666 374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:00:34.221728 374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:00:34.221783 377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:34.221969 377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:34.223438 377 catalog_manager.cc:1347] Generated new cluster ID: 2f277b61660b4e319d8daa1e4d507307
I20260808 07:00:34.223493 377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:34.226841 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.220813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.24.181.126:37303)
0808 07:00:34.221047 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.221055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.221423 (+ 368us) server_negotiation.cc:407] Connection header received
0808 07:00:34.221648 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.221654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.221713 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.221880 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:34.222234 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.223261 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.224448 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.225065 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.225174 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.226051 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:34.226056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:34.226056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:34.226250 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:34.226288 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:34.226301 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:34.226351 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:34.226548 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 07:00:34.226621 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260808 07:00:34.228331 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:00:34.240460 377 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:34.240994 377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:34.244697 377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5042567308848c2a6bd5c30deee8734: Generated new TSK 0
I20260808 07:00:34.244889 377 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:00:34.266808 390 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 07:00:34.267001 390 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 07:00:34.267445 389 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:34.251592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:37303 (local address 127.0.0.1:34754)
0808 07:00:34.251793 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:34.251803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:34.251887 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:34.252093 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:34.252094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:34.252096 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:00:34.252169 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.252173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.252850 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.252851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.253440 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.253442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.253816 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.253818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.253885 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.253913 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:00:34.267095 (+ 13182us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:00:34.267209 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.181.126:37303: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260808 07:00:34.267794 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.251661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.24.181.126:37303)
0808 07:00:34.251708 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.251710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.251865 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:00:34.251985 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.251986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.252000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.252082 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:00:34.252253 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.252762 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.253529 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.253731 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.253811 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.254023 (+ 212us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:00:34.266909 (+ 12886us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:00:34.267689 (+ 780us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:00:34.267848 370 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.181.126:37303: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34683/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 07:00:34.268081 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 32725
2026-08-08T07:00:34Z chronyd exiting
I20260808 07:00:34.311919 25301 test_util.cc:175] -----------------------------------------------
I20260808 07:00:34.311997 25301 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172407597382-25301-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (237 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 07:00:34.312764 25301 mini_oidc.cc:152] Starting JWKS server
I20260808 07:00:34.313464 25301 webserver.cc:460] Webserver started at https://127.0.0.1:36293/ using document root <none> and password file <none>
I20260808 07:00:34.313504 25301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:00:34.313696 25301 webserver.cc:460] Webserver started at http://127.0.0.1:35917/ using document root <none> and password file <none>
2026-08-08T07:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:34Z Disabled control of system clock
I20260808 07:00:34.322049 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46671
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:46671
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36293/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/testchainca.cert with env {}
W20260808 07:00:34.402880 401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:34.403172 401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:34.403208 401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:34.404371 401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:34.404445 401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:34.404472 401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:34.404505 401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:34.404529 401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:00:34.404558 401 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:00:34.404584 401 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36293/jwks.json
I20260808 07:00:34.406124 401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:46671
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36293/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:34.406409 401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:34.406625 401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:34.411471 408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:34.411491 407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:34.411471 411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.411598 401 server_base.cc:1034] running on GCE node
I20260808 07:00:34.411959 401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:34.412245 401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:34.413414 401 hybrid_clock.cc:648] HybridClock initialized: now 1786172434413413 us; error 56 us; skew 500 ppm
I20260808 07:00:34.419392 401 webserver.cc:460] Webserver started at http://127.24.181.126:46025/ using document root <none> and password file <none>
I20260808 07:00:34.419649 401 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:34.419723 401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:34.419878 401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:34.420938 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "8e7bf9864a6a4e1a865a241bcc71f84d"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ad3cb59f8b699dd1da9cbdc22174f1f4"
server_key_iv: "e3e7fbadf6e19277a3745665de6e3955"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.421326 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "8e7bf9864a6a4e1a865a241bcc71f84d"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ad3cb59f8b699dd1da9cbdc22174f1f4"
server_key_iv: "e3e7fbadf6e19277a3745665de6e3955"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.422613 401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:00:34.423416 421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:34.423584 401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:34.423694 401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "8e7bf9864a6a4e1a865a241bcc71f84d"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ad3cb59f8b699dd1da9cbdc22174f1f4"
server_key_iv: "e3e7fbadf6e19277a3745665de6e3955"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.423769 401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:34.450793 401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:34.451200 401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:34.451377 401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:34.455340 401 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:46671
I20260808 07:00:34.455397 499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:46671 every 8 connection(s)
I20260808 07:00:34.455773 401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:34.456306 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 401
I20260808 07:00:34.456425 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:34.456589 502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:34.456616 25301 external_mini_cluster.cc:1428] Setting key 87169fb5a143b7fbf0b697e80b5edbde
I20260808 07:00:34.460553 502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d: Bootstrap starting.
I20260808 07:00:34.461194 502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:34.461500 502 log.cc:824] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:34.462215 502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d: No bootstrap required, opened a new log
I20260808 07:00:34.462716 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.457233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.24.181.126:46671)
0808 07:00:34.457473 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.457482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.457516 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:00:34.458050 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.458060 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.458164 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.458322 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:34.458538 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.459902 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.460725 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.461156 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.461309 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.462044 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:34.462048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:34.462048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:34.462232 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:34.462293 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:34.462309 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:34.462390 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:34.462467 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:00:34.462548 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260808 07:00:34.463881 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:00:34.464675 502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } }
I20260808 07:00:34.464875 502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:34.464928 502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7bf9864a6a4e1a865a241bcc71f84d, State: Initialized, Role: FOLLOWER
I20260808 07:00:34.465054 502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } }
I20260808 07:00:34.465163 502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:34.465197 502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:34.465255 502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:34.465824 502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } }
I20260808 07:00:34.465915 502 leader_election.cc:302] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e7bf9864a6a4e1a865a241bcc71f84d; no voters:
I20260808 07:00:34.466097 502 leader_election.cc:288] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:34.466207 507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:34.466346 502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:34.466384 507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [term 1 LEADER]: Becoming Leader. State: Replica: 8e7bf9864a6a4e1a865a241bcc71f84d, State: Running, Role: LEADER
I20260808 07:00:34.466482 507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } }
I20260808 07:00:34.466905 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } } }
I20260808 07:00:34.467038 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.466907 510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e7bf9864a6a4e1a865a241bcc71f84d. Latest consensus state: current_term: 1 leader_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e7bf9864a6a4e1a865a241bcc71f84d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46671 } } }
I20260808 07:00:34.467239 518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:34.467259 510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.467854 518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:34.469319 518 catalog_manager.cc:1347] Generated new cluster ID: 720ebee7adae4a7e8e590720b61e492b
I20260808 07:00:34.469377 518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:34.477089 518 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:34.477573 518 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:34.483532 518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e7bf9864a6a4e1a865a241bcc71f84d: Generated new TSK 0
I20260808 07:00:34.483724 518 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:00:34.508111 536 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 07:00:34.508214 536 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:00:34.508543 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.489801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33346 (local address 127.24.181.126:46671)
0808 07:00:34.489850 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.489852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.489973 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:00:34.490033 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.490034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.490047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.490106 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:00:34.490263 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.490745 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.491918 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.492129 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.492184 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.492367 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:00:34.508358 (+ 15991us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:00:34.508489 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 07:00:34.508615 505 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 07:00:34.508739 535 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:34.489679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46671 (local address 127.0.0.1:33346)
0808 07:00:34.489879 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:34.489891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:34.489961 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:34.490124 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:34.490125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:34.490127 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:00:34.490196 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.490199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.490840 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.490842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.491838 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.491841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.492206 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.492207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.492269 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.492274 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:00:34.508482 (+ 16208us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:00:34.508632 (+ 150us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.181.126:46671: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
I20260808 07:00:34.509073 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 401
2026-08-08T07:00:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 07:00:34.557547 25301 mini_oidc.cc:152] Starting JWKS server
I20260808 07:00:34.558270 25301 webserver.cc:460] Webserver started at https://127.0.0.1:36801/ using document root <none> and password file <none>
I20260808 07:00:34.558346 25301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:00:34.558537 25301 webserver.cc:460] Webserver started at http://127.0.0.1:39241/ using document root <none> and password file <none>
2026-08-08T07:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:34Z Disabled control of system clock
I20260808 07:00:34.567381 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33039
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:33039
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36801/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:00:34.647745 551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:34.647948 551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:34.647969 551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:34.649153 551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:34.649199 551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:34.649211 551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:34.649224 551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:34.649236 551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:00:34.649250 551 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:00:34.649260 551 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36801/jwks.json
I20260808 07:00:34.650612 551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:33039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36801/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:34.650863 551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:34.651212 551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:34.656128 558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:34.656138 556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.656239 551 server_base.cc:1034] running on GCE node
W20260808 07:00:34.656128 562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.656620 551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:34.656895 551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:34.658006 551 hybrid_clock.cc:648] HybridClock initialized: now 1786172434658001 us; error 41 us; skew 500 ppm
I20260808 07:00:34.659286 551 webserver.cc:460] Webserver started at http://127.24.181.126:42719/ using document root <none> and password file <none>
I20260808 07:00:34.659528 551 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:34.659590 551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:34.659749 551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:34.660823 551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "56349015da714160baa9c4ba14bdcae2"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ef47d7ff7f798dc4d90a6e8cb09cd170"
server_key_iv: "4efedabe5e3b49ed55bc5466d9494129"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.661230 551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "56349015da714160baa9c4ba14bdcae2"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ef47d7ff7f798dc4d90a6e8cb09cd170"
server_key_iv: "4efedabe5e3b49ed55bc5466d9494129"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.662570 551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:34.663400 569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:34.663581 551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:34.663682 551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "56349015da714160baa9c4ba14bdcae2"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "ef47d7ff7f798dc4d90a6e8cb09cd170"
server_key_iv: "4efedabe5e3b49ed55bc5466d9494129"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.663761 551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:34.673734 551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:34.674019 551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:34.674165 551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:34.678061 551 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:33039
I20260808 07:00:34.678115 639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:33039 every 8 connection(s)
I20260808 07:00:34.678455 551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:34.679188 640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:34.681488 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 551
I20260808 07:00:34.681595 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:34.681777 25301 external_mini_cluster.cc:1428] Setting key c56dfdd55553a7eef32044a69ab6fb5a
I20260808 07:00:34.681782 640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2: Bootstrap starting.
I20260808 07:00:34.682559 640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:34.682883 640 log.cc:824] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:34.683665 640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2: No bootstrap required, opened a new log
I20260808 07:00:34.685910 640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } }
I20260808 07:00:34.686074 640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:34.686110 640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56349015da714160baa9c4ba14bdcae2, State: Initialized, Role: FOLLOWER
I20260808 07:00:34.686242 640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } }
I20260808 07:00:34.686317 640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:34.686352 640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:34.686403 640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:34.687224 640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } }
I20260808 07:00:34.687335 640 leader_election.cc:302] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56349015da714160baa9c4ba14bdcae2; no voters:
I20260808 07:00:34.687507 640 leader_election.cc:288] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:34.687536 646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:34.687706 646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [term 1 LEADER]: Becoming Leader. State: Replica: 56349015da714160baa9c4ba14bdcae2, State: Running, Role: LEADER
I20260808 07:00:34.687769 640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:34.687796 646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } }
I20260808 07:00:34.688060 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.682282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.24.181.126:33039)
0808 07:00:34.682507 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.682515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.682552 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:00:34.683218 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.683225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.683290 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.683454 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:34.683626 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.685011 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.685756 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.686358 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.686481 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.687219 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:34.687223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:34.687223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:34.687437 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:34.687483 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:34.687499 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:34.687578 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:34.687665 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:00:34.687856 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 07:00:34.688189 646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56349015da714160baa9c4ba14bdcae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } } }
I20260808 07:00:34.688257 646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.688383 646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56349015da714160baa9c4ba14bdcae2. Latest consensus state: current_term: 1 leader_uuid: "56349015da714160baa9c4ba14bdcae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56349015da714160baa9c4ba14bdcae2" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33039 } } }
I20260808 07:00:34.688442 646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.688529 653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:34.688942 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:00:34.689073 653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:34.690810 653 catalog_manager.cc:1347] Generated new cluster ID: 7c1cfe8cef914e28b0e2d8c8779fbbbd
I20260808 07:00:34.690887 653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:34.702131 653 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:34.702605 653 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:34.709129 653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56349015da714160baa9c4ba14bdcae2: Generated new TSK 0
I20260808 07:00:34.709313 653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:34.722426 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.718855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.24.181.126:33039)
0808 07:00:34.718934 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.718936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.719051 (+ 115us) server_negotiation.cc:407] Connection header received
0808 07:00:34.719117 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.719118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.719127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.719187 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:00:34.719320 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.719832 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.720996 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.721186 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.721242 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.721420 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:00:34.722200 (+ 780us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:00:34.722353 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:00:34.722488 643 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48362: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 07:00:34.722506 681 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:34.718798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33039 (local address 127.0.0.1:48362)
0808 07:00:34.718971 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:34.718984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:34.719053 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:34.719187 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:34.719188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:34.719190 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:00:34.719259 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.719261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.719915 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.719916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.720906 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.720909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.721266 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.721267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.721326 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.721331 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:00:34.722306 (+ 975us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:00:34.722405 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.181.126:33039: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260808 07:00:34.722877 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 551
E20260808 07:00:34.736474 682 webserver.cc:533] Webserver: sslize failed:
E20260808 07:00:34.736545 682 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T07:00:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T07:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:34Z Disabled control of system clock
I20260808 07:00:34.774219 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44939
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44939
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/testchainca.cert with env {}
W20260808 07:00:34.858178 692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:34.858418 692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:34.858457 692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:34.859715 692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:34.859781 692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:34.859803 692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:34.859838 692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:34.859886 692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:00:34.859917 692 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:00:34.859946 692 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 07:00:34.861487 692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:34.861773 692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:34.861999 692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:34.867087 701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.867134 692 server_base.cc:1034] running on GCE node
W20260808 07:00:34.867077 699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:34.867089 705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:34.867596 692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:34.867969 692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:34.869155 692 hybrid_clock.cc:648] HybridClock initialized: now 1786172434869131 us; error 38 us; skew 500 ppm
I20260808 07:00:34.870375 692 webserver.cc:460] Webserver started at http://127.24.181.126:33839/ using document root <none> and password file <none>
I20260808 07:00:34.870633 692 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:34.870769 692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:34.870967 692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:34.872016 692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c522bdf12edd4ddaa6a3b97b8e334027"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "93545e625af5891b320e57ef4dbc13ab"
server_key_iv: "c72c204a78a29a1dc05a204f04639811"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.872421 692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c522bdf12edd4ddaa6a3b97b8e334027"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "93545e625af5891b320e57ef4dbc13ab"
server_key_iv: "c72c204a78a29a1dc05a204f04639811"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.873775 692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:34.874511 715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:34.874722 692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:34.874827 692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c522bdf12edd4ddaa6a3b97b8e334027"
format_stamp: "Formatted at 2026-08-08 07:00:34 on dist-test-slave-nfb5"
server_key: "93545e625af5891b320e57ef4dbc13ab"
server_key_iv: "c72c204a78a29a1dc05a204f04639811"
server_key_version: "encryptionkey@0"
I20260808 07:00:34.874912 692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:34.884930 692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:34.885293 692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:34.885452 692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:34.889585 692 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44939
I20260808 07:00:34.889616 801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44939 every 8 connection(s)
I20260808 07:00:34.889962 692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:34.891055 803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:34.893278 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027: Bootstrap starting.
I20260808 07:00:34.893808 803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:34.894042 803 log.cc:824] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:34.894685 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027: No bootstrap required, opened a new log
I20260808 07:00:34.896275 803 raft_consensus.cc:348] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } }
I20260808 07:00:34.896420 803 raft_consensus.cc:374] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:34.896445 803 raft_consensus.cc:729] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c522bdf12edd4ddaa6a3b97b8e334027, State: Initialized, Role: FOLLOWER
I20260808 07:00:34.896543 803 consensus_queue.cc:260] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } }
I20260808 07:00:34.896621 803 raft_consensus.cc:388] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:34.896661 803 raft_consensus.cc:482] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:34.896734 803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:34.897257 803 raft_consensus.cc:504] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } }
I20260808 07:00:34.897385 803 leader_election.cc:302] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c522bdf12edd4ddaa6a3b97b8e334027; no voters:
I20260808 07:00:34.897553 803 leader_election.cc:288] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:34.897634 806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:34.897790 806 raft_consensus.cc:686] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [term 1 LEADER]: Becoming Leader. State: Replica: c522bdf12edd4ddaa6a3b97b8e334027, State: Running, Role: LEADER
I20260808 07:00:34.897802 803 sys_catalog.cc:564] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:34.897895 806 consensus_queue.cc:237] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } }
I20260808 07:00:34.898249 807 sys_catalog.cc:455] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } } }
I20260808 07:00:34.898379 808 sys_catalog.cc:455] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c522bdf12edd4ddaa6a3b97b8e334027. Latest consensus state: current_term: 1 leader_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c522bdf12edd4ddaa6a3b97b8e334027" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44939 } } }
I20260808 07:00:34.898447 808 sys_catalog.cc:458] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.898409 807 sys_catalog.cc:458] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:34.898666 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 692
I20260808 07:00:34.898715 817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:34.898777 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:34.899019 25301 external_mini_cluster.cc:1428] Setting key b97e744870dfa33118247dc567963981
I20260808 07:00:34.899308 817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:34.901767 817 catalog_manager.cc:1347] Generated new cluster ID: 03f9c1bff86c4e1997768d752c86ea5b
I20260808 07:00:34.901829 817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:34.904914 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.899638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.24.181.126:44939)
0808 07:00:34.899886 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.899898 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.899936 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:00:34.900092 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.900099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.900168 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.900341 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:34.900605 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.901578 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.902673 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.903357 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.903490 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.904187 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:34.904191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:34.904192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:00:34.904424 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:34.904469 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:00:34.904483 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:34.904557 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:34.904627 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:00:34.904706 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":116,"threads_started":1}
I20260808 07:00:34.905988 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:00:34.918864 817 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:34.919381 817 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:34.924921 817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c522bdf12edd4ddaa6a3b97b8e334027: Generated new TSK 0
I20260808 07:00:34.925104 817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:35.361382 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:34.934250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.24.181.126:44939)
0808 07:00:34.934301 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:34.934303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:34.934410 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:00:34.934490 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:34.934491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:34.934501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:34.934590 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:00:34.934831 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.935415 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.936249 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.936448 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.936524 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.936722 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:00:35.361059 (+424337us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:00:35.361287 (+ 228us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41718: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 07:00:35.361472 834 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41718: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 07:00:35.361491 844 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:34.934143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44939 (local address 127.0.0.1:41718)
0808 07:00:34.934334 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:34.934346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:34.934417 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:34.934664 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:34.934665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:34.934667 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:00:34.934751 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.934753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.935503 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.935504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.936129 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:34.936131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:34.936529 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:34.936531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:34.936598 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:34.936603 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:00:35.361229 (+424626us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:00:35.361363 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.181.126:44939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260808 07:00:35.361845 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 692
2026-08-08T07:00:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (619 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:35 dist-test-slave-nfb5 krb5kdc[858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:35 dist-test-slave-nfb5 krb5kdc[858](info): set up 2 sockets
Aug 08 07:00:35 dist-test-slave-nfb5 krb5kdc[858](info): commencing operation
krb5kdc: starting...
W20260808 07:00:36.768923 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:36 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:36.894208 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43075
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:36.976028 882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:36.976238 882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:36.976261 882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:36.977358 882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:36.977394 882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:36.977408 882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:36.977421 882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:36.977434 882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:36.978825 882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:36.979091 882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:36.979276 882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:36.984118 890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:36.984128 889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:36.984184 882 server_base.cc:1034] running on GCE node
W20260808 07:00:36.984118 894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:36.984740 882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:36.985031 882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:36.986212 882 hybrid_clock.cc:648] HybridClock initialized: now 1786172436986191 us; error 37 us; skew 500 ppm
Aug 08 07:00:36 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172436, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:36.989173 882 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:36.989606 882 webserver.cc:460] Webserver started at http://127.24.181.126:41327/ using document root <none> and password file <none>
I20260808 07:00:36.989842 882 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:36.989912 882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:36.990052 882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:36.991163 882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c948b5f4a2e64df5bd35e9ae253577f8"
format_stamp: "Formatted at 2026-08-08 07:00:36 on dist-test-slave-nfb5"
server_key: "323a18b9b0f0bcc1fb37aaaf3409918e"
server_key_iv: "28bbe1fa1b4e2242acfa6fdc02d139b5"
server_key_version: "encryptionkey@0"
I20260808 07:00:36.991532 882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c948b5f4a2e64df5bd35e9ae253577f8"
format_stamp: "Formatted at 2026-08-08 07:00:36 on dist-test-slave-nfb5"
server_key: "323a18b9b0f0bcc1fb37aaaf3409918e"
server_key_iv: "28bbe1fa1b4e2242acfa6fdc02d139b5"
server_key_version: "encryptionkey@0"
I20260808 07:00:36.992914 882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:36.993641 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:36.993813 882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:36.993918 882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c948b5f4a2e64df5bd35e9ae253577f8"
format_stamp: "Formatted at 2026-08-08 07:00:36 on dist-test-slave-nfb5"
server_key: "323a18b9b0f0bcc1fb37aaaf3409918e"
server_key_iv: "28bbe1fa1b4e2242acfa6fdc02d139b5"
server_key_version: "encryptionkey@0"
I20260808 07:00:36.994004 882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:37.002252 882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:37.004591 882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:37.004757 882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:37.008778 882 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:43075
I20260808 07:00:37.008839 985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:43075 every 8 connection(s)
I20260808 07:00:37.009200 882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:37.010098 986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:37.012487 986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8: Bootstrap starting.
I20260808 07:00:37.013056 986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:37.013375 986 log.cc:824] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:37.014048 986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8: No bootstrap required, opened a new log
I20260808 07:00:37.015719 986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } }
I20260808 07:00:37.015875 986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:37.015921 986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c948b5f4a2e64df5bd35e9ae253577f8, State: Initialized, Role: FOLLOWER
I20260808 07:00:37.016084 986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } }
I20260808 07:00:37.016186 986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:37.016228 986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:37.016289 986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:37.016804 986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } }
I20260808 07:00:37.016921 986 leader_election.cc:302] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c948b5f4a2e64df5bd35e9ae253577f8; no voters:
I20260808 07:00:37.017107 986 leader_election.cc:288] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:37.017154 993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:37.017366 986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:37.017558 993 raft_consensus.cc:686] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [term 1 LEADER]: Becoming Leader. State: Replica: c948b5f4a2e64df5bd35e9ae253577f8, State: Running, Role: LEADER
I20260808 07:00:37.017652 993 consensus_queue.cc:237] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } }
I20260808 07:00:37.018064 999 sys_catalog.cc:455] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c948b5f4a2e64df5bd35e9ae253577f8. Latest consensus state: current_term: 1 leader_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } } }
I20260808 07:00:37.018067 998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c948b5f4a2e64df5bd35e9ae253577f8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43075 } } }
I20260808 07:00:37.018174 999 sys_catalog.cc:458] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:37.018181 998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:37.018443 1011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:37.019016 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 882
I20260808 07:00:37.019064 1011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:37.019133 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:37.019345 25301 external_mini_cluster.cc:1428] Setting key 181032939ada96ebd11d80851e23bba4
I20260808 07:00:37.020750 1011 catalog_manager.cc:1347] Generated new cluster ID: ff90a45afa1b4757ba9d1ec46d9f95c8
I20260808 07:00:37.020812 1011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:37.028086 1011 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:37.028756 1011 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:37.031601 1011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c948b5f4a2e64df5bd35e9ae253577f8: Generated new TSK 0
I20260808 07:00:37.032074 1011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:37.034766 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.019810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.24.181.126:43075)
0808 07:00:37.023361 (+ 3551us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.023371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.023402 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:00:37.023521 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.023528 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.023593 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.023746 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.024515 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.026542 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.027622 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.028265 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.028380 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.031464 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.031468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.031469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.031480 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.033212 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.033629 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.033632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.033632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.033719 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.033925 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.033925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.033926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.034124 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.034230 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.034369 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.034488 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3497,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:37.061353 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:37.143903 1031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:37.144145 1031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:37.144182 1031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:37.145349 1031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:37.145419 1031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:37.145465 1031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:37.147037 1031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:37.147305 1031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:37.147552 1031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:37.152661 1041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.152683 1037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.152776 1038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:37.152927 1031 server_base.cc:1034] running on GCE node
I20260808 07:00:37.153167 1031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:37.153417 1031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:37.154568 1031 hybrid_clock.cc:648] HybridClock initialized: now 1786172437154558 us; error 33 us; skew 500 ppm
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.157105 1031 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:37.157539 1031 webserver.cc:460] Webserver started at http://127.24.181.65:34825/ using document root <none> and password file <none>
I20260808 07:00:37.157783 1031 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:37.157855 1031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:37.158026 1031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:37.159210 1031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8ee8c2d05db84aa2b702b2d4dfaf8df4"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ed0449ae97e398af3f715939c420b06f"
server_key_iv: "59fa165c3b04005c90ec448167545087"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.159592 1031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8ee8c2d05db84aa2b702b2d4dfaf8df4"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ed0449ae97e398af3f715939c420b06f"
server_key_iv: "59fa165c3b04005c90ec448167545087"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.160945 1031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:00:37.161743 1049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.161939 1031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:37.162050 1031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8ee8c2d05db84aa2b702b2d4dfaf8df4"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ed0449ae97e398af3f715939c420b06f"
server_key_iv: "59fa165c3b04005c90ec448167545087"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.162129 1031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:37.186049 1031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:37.188633 1031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:37.188794 1031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:37.189073 1031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:37.189445 1031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:37.189494 1031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.189531 1031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:37.189553 1031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.196887 1031 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:43609
I20260808 07:00:37.196946 1233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:43609 every 8 connection(s)
I20260808 07:00:37.197264 1031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:37.207072 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.198019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:41461 (local address 127.24.181.126:43075)
0808 07:00:37.198101 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.198104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.198451 (+ 347us) server_negotiation.cc:407] Connection header received
0808 07:00:37.199481 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.199482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.199496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.199546 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.200199 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.200726 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.201729 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.201965 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.202055 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.204300 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.204302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.204302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.204310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.205551 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.206040 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.206042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.206042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.206110 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.206378 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.206379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.206379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.206501 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.206572 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.206858 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.206976 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:00:37.207129 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 1031
I20260808 07:00:37.207217 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:37.207280 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.198008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43075 (local address 127.24.181.65:41461)
0808 07:00:37.198311 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:37.198353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:37.199346 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:37.199624 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:37.199628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:37.199889 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:37.200081 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.200085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.200830 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.200831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.201629 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.201631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.202069 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.202070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.202197 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.202714 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:37.202723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:37.204192 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:37.205709 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.205717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.205720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.205949 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.206185 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.206186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.206187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.206300 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.206576 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:37.206584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:37.206717 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 07:00:37.206823 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:00:37.207014 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":195,"threads_started":1}
I20260808 07:00:37.207495 25301 external_mini_cluster.cc:1428] Setting key c72e6384bdc9b285155b7313ee0a9a45
I20260808 07:00:37.208134 1234 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43075
I20260808 07:00:37.208254 1234 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:37.208444 1234 heartbeater.cc:511] Master 127.24.181.126:43075 requested a full tablet report, sending...
I20260808 07:00:37.208956 936 ts_manager.cc:194] Registered new tserver with Master: 8ee8c2d05db84aa2b702b2d4dfaf8df4 (127.24.181.65:43609)
I20260808 07:00:37.209571 936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:41461
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:37.233386 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:37.317441 1244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:37.317706 1244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:37.317742 1244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:37.319013 1244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:37.319053 1244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:37.319080 1244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:37.320539 1244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:37.320796 1244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:37.321022 1244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:37.326063 1258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.326139 1255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.326063 1254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:37.326287 1244 server_base.cc:1034] running on GCE node
I20260808 07:00:37.326498 1244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:37.326738 1244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:37.327883 1244 hybrid_clock.cc:648] HybridClock initialized: now 1786172437327867 us; error 37 us; skew 500 ppm
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.330483 1244 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:37.331044 1244 webserver.cc:460] Webserver started at http://127.24.181.66:36937/ using document root <none> and password file <none>
I20260808 07:00:37.331292 1244 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:37.331359 1244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:37.331498 1244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:37.332525 1244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "52358dfd5a664de09daf3737237ca8f8"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ab23a32cec5af58af280c2b859b7a99f"
server_key_iv: "95caec0496f0088b79a613c171ac4514"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.332893 1244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "52358dfd5a664de09daf3737237ca8f8"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ab23a32cec5af58af280c2b859b7a99f"
server_key_iv: "95caec0496f0088b79a613c171ac4514"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.334268 1244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:37.335091 1268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.335275 1244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:37.335371 1244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "52358dfd5a664de09daf3737237ca8f8"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "ab23a32cec5af58af280c2b859b7a99f"
server_key_iv: "95caec0496f0088b79a613c171ac4514"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.335454 1244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:37.348238 1244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:37.350421 1244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:37.350591 1244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:37.350843 1244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:37.351178 1244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:37.351233 1244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.351267 1244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:37.351300 1244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.357465 1244 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:36353
I20260808 07:00:37.357514 1452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:36353 every 8 connection(s)
I20260808 07:00:37.357813 1244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:37.359735 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 1244
I20260808 07:00:37.359863 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:37.360046 25301 external_mini_cluster.cc:1428] Setting key 81098906c670dfa0d8aae892739d83b5
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:37.368592 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.358571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:47857 (local address 127.24.181.126:43075)
0808 07:00:37.358637 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.358640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.358865 (+ 225us) server_negotiation.cc:407] Connection header received
0808 07:00:37.359849 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.359851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.359866 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.359918 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.360631 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.361371 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.362341 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.362651 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.362710 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.365556 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.365557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.365557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.365564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.367077 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.367579 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.367580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.367580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.367632 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.367930 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.367931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.367932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.368055 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.368110 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.368462 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.368527 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:00:37.368880 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.358545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43075 (local address 127.24.181.66:47857)
0808 07:00:37.358708 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:37.358750 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:37.359716 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:37.359995 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:37.359998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:37.360298 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:37.360541 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.360546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.361465 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.361465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.362251 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.362252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.362743 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.362744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.362907 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.363508 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:37.363521 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:37.365446 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:37.367159 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.367167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.367170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.367498 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.367692 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.367693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.367694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.367845 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.368131 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:37.368140 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:37.368314 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 07:00:37.368444 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 07:00:37.368618 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 07:00:37.369412 1453 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43075
I20260808 07:00:37.369504 1453 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:37.369676 1453 heartbeater.cc:511] Master 127.24.181.126:43075 requested a full tablet report, sending...
I20260808 07:00:37.370062 936 ts_manager.cc:194] Registered new tserver with Master: 52358dfd5a664de09daf3737237ca8f8 (127.24.181.66:36353)
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
I20260808 07:00:37.370409 936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:47857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:37.389950 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:37.474478 1459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:37.474730 1459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:37.474772 1459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:37.476011 1459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:37.476085 1459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:37.476133 1459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:37.477680 1459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:37.477962 1459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:37.478206 1459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:37.483475 1468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.483476 1472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:37.483476 1469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:37.483494 1459 server_base.cc:1034] running on GCE node
I20260808 07:00:37.483981 1459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:37.484236 1459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:37.485414 1459 hybrid_clock.cc:648] HybridClock initialized: now 1786172437485404 us; error 34 us; skew 500 ppm
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.488206 1459 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:37.488669 1459 webserver.cc:460] Webserver started at http://127.24.181.67:37801/ using document root <none> and password file <none>
I20260808 07:00:37.488875 1459 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:37.488917 1459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:37.489010 1459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:37.490010 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "e327f8fc7dba4b83b967697f3d6cb8ae"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "793a5d7eb04df8ad384bc2ff7a118b8a"
server_key_iv: "cb4baf942127be323453c0386c36a35f"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.490334 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "e327f8fc7dba4b83b967697f3d6cb8ae"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "793a5d7eb04df8ad384bc2ff7a118b8a"
server_key_iv: "cb4baf942127be323453c0386c36a35f"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.491745 1459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:37.492453 1483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.492648 1459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:37.492731 1459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "e327f8fc7dba4b83b967697f3d6cb8ae"
format_stamp: "Formatted at 2026-08-08 07:00:37 on dist-test-slave-nfb5"
server_key: "793a5d7eb04df8ad384bc2ff7a118b8a"
server_key_iv: "cb4baf942127be323453c0386c36a35f"
server_key_version: "encryptionkey@0"
I20260808 07:00:37.492787 1459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:37.514487 1459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:37.516907 1459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:37.517098 1459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:37.517299 1459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:37.517669 1459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:37.517707 1459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.517730 1459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:37.517771 1459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:37.523790 1459 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:38289
I20260808 07:00:37.523872 1647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:38289 every 8 connection(s)
I20260808 07:00:37.524195 1459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:37.525324 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 1459
I20260808 07:00:37.525441 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:37.525622 25301 external_mini_cluster.cc:1428] Setting key 531077549a67d2871261e8d5503ba1a0
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:37.534047 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.524894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:39265 (local address 127.24.181.126:43075)
0808 07:00:37.524937 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.524939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.525179 (+ 240us) server_negotiation.cc:407] Connection header received
0808 07:00:37.526508 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.526509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.526520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.526577 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.527331 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.528173 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.529152 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.529344 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.529398 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.531505 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.531521 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.531521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.531529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.532909 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.533268 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.533269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.533269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.533325 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.533534 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.533534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.533534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.533631 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.533681 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.533922 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.533984 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:00:37.534269 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.524867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43075 (local address 127.24.181.67:39265)
0808 07:00:37.525030 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:37.525073 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:37.526413 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:37.526661 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:37.526666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:37.527007 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:37.527230 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.527236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.528279 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.528280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.529076 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:37.529078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.529420 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.529421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:37.529519 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.529954 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:37.529962 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:37.531414 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:37.532982 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.532988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.532990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.533204 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.533383 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:37.533383 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:37.533384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:37.533481 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:37.533685 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:37.533689 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:37.533814 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:00:37.533897 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:00:37.534016 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260808 07:00:37.534747 1648 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43075
I20260808 07:00:37.534854 1648 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:37.535059 1648 heartbeater.cc:511] Master 127.24.181.126:43075 requested a full tablet report, sending...
I20260808 07:00:37.535393 936 ts_manager.cc:194] Registered new tserver with Master: e327f8fc7dba4b83b967697f3d6cb8ae (127.24.181.67:38289)
I20260808 07:00:37.535774 936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:39265
I20260808 07:00:37.537403 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/test-admin.keytab.
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.555361 25301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 07:00:37.557147 1658 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:37.556297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.24.181.65:43609)
0808 07:00:37.556472 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.556477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.556493 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:00:37.556571 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.556573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.556586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.556662 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.557033 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56296: BlockingRecv error: recv got EOF from 127.0.0.1:56296 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260808 07:00:37.557319 25301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.558300 25301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.559197 25301 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:37.567104 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.559682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.24.181.65:43609)
0808 07:00:37.559735 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.559737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.559760 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:00:37.559833 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.559834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.559843 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.559895 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.560296 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.561039 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.561759 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.562068 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.562124 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.564482 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.564490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.564491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.564497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.566079 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.566401 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.566404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.566404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.566459 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.566624 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.566625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.566625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.566748 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.566860 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.566960 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.567033 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 07:00:37.569146 1658 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:37.568418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.24.181.65:43609)
0808 07:00:37.568476 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.568478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.568571 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:00:37.568644 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.568644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.568656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.568704 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.569092 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56312: BlockingRecv error: recv got EOF from 127.0.0.1:56312 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:00:37.569252 25301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.570258 25301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.571265 25301 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:37.578437 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.571765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.24.181.65:43609)
0808 07:00:37.571821 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.571823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.571925 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:00:37.571996 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.571997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.572008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.572056 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.572415 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.572894 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.573587 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.573798 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.573873 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.575994 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.575995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.575995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.576002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.577296 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.577655 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.577657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.577657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.577719 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.577932 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.577933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.577933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.578036 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.578140 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.578249 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.578340 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:00:37.579945 1658 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:37.579410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.24.181.65:43609)
0808 07:00:37.579471 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.579473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.579493 (+ 20us) server_negotiation.cc:407] Connection header received
0808 07:00:37.579550 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.579550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.579557 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.579600 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.579895 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56330: BlockingRecv error: recv got EOF from 127.0.0.1:56330 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 07:00:37.580015 25301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172407597382-25301-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.581072 25301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:37.581976 25301 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:00:37.588497 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:37.582499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.24.181.65:43609)
0808 07:00:37.582539 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:37.582541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:37.582570 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:00:37.582620 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:37.582620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:37.582627 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:37.582671 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:37.583090 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.583541 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.584194 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:37.584372 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:37.584421 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:37.586365 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:37.586366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:37.586367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:37.586373 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:37.587611 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.587933 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.587934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.587934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.587981 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:37.588140 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:37.588141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:37.588141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:37.588221 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:37.588290 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:37.588389 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:00:37.588442 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:00:37.589093 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 1031
I20260808 07:00:37.602038 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 1244
I20260808 07:00:37.623512 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 1459
I20260808 07:00:37.633463 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 882
2026-08-08T07:00:37Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2095 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[1675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[1675](info): set up 2 sockets
Aug 08 07:00:37 dist-test-slave-nfb5 krb5kdc[1675](info): commencing operation
krb5kdc: starting...
W20260808 07:00:38.874505 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:38 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab.
Entry for principal oryx/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab.
I20260808 07:00:38.997875 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43181
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab
--principal=oryx/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:39.081123 1694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:39.081327 1694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:39.081351 1694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:39.082427 1694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:39.082461 1694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:39.082475 1694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:39.082489 1694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:39.082501 1694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:39.084019 1694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab
--principal=oryx/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:39.084282 1694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:39.084518 1694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.089499 1700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.089525 1701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.089525 1694 server_base.cc:1034] running on GCE node
W20260808 07:00:39.089499 1704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.089978 1694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.090323 1694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.091471 1694 hybrid_clock.cc:648] HybridClock initialized: now 1786172439091452 us; error 40 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.094055 1694 init.cc:377] Logged in from keytab as oryx/127.24.181.126@KRBTEST.COM (short username oryx)
I20260808 07:00:39.094473 1694 webserver.cc:460] Webserver started at http://127.24.181.126:39765/ using document root <none> and password file <none>
I20260808 07:00:39.094660 1694 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.094702 1694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.094794 1694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:39.095883 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "ec45ae6459b4465bb4f3616db3eb32b3"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "0b6365475f5a1e80d72ce54c2573aa19"
server_key_iv: "0f8949df6e046745c37194c7380e8772"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.096192 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "ec45ae6459b4465bb4f3616db3eb32b3"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "0b6365475f5a1e80d72ce54c2573aa19"
server_key_iv: "0f8949df6e046745c37194c7380e8772"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.097527 1694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:39.098248 1712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.098464 1694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:39.098578 1694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "ec45ae6459b4465bb4f3616db3eb32b3"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "0b6365475f5a1e80d72ce54c2573aa19"
server_key_iv: "0f8949df6e046745c37194c7380e8772"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.098660 1694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.108572 1694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.110510 1694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:39.110685 1694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.114830 1694 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:43181
I20260808 07:00:39.114881 1788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:43181 every 8 connection(s)
I20260808 07:00:39.115272 1694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:39.116292 1789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:39.118501 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Bootstrap starting.
I20260808 07:00:39.119093 1789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:39.119342 1789 log.cc:824] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:39.120038 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: No bootstrap required, opened a new log
I20260808 07:00:39.121627 1789 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } }
I20260808 07:00:39.121765 1789 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:39.121788 1789 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec45ae6459b4465bb4f3616db3eb32b3, State: Initialized, Role: FOLLOWER
I20260808 07:00:39.121929 1789 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } }
I20260808 07:00:39.122014 1789 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:39.122037 1789 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:39.122067 1789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:39.122828 1789 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } }
I20260808 07:00:39.122967 1789 leader_election.cc:302] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec45ae6459b4465bb4f3616db3eb32b3; no voters:
I20260808 07:00:39.123174 1789 leader_election.cc:288] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:39.123219 1793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:39.123256 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 1694
I20260808 07:00:39.123394 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:39.123415 1793 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 LEADER]: Becoming Leader. State: Replica: ec45ae6459b4465bb4f3616db3eb32b3, State: Running, Role: LEADER
I20260808 07:00:39.123440 1789 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:39.123504 1793 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } }
I20260808 07:00:39.123651 25301 external_mini_cluster.cc:1428] Setting key 21494f6d757034aafd06cf660f598033
I20260808 07:00:39.123872 1793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec45ae6459b4465bb4f3616db3eb32b3. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } }
I20260808 07:00:39.123961 1793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:39.124150 1795 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } }
I20260808 07:00:39.124223 1795 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:39.124380 1805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:39.124913 1805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:39.127142 1805 catalog_manager.cc:1347] Generated new cluster ID: 062349e0c0964164a4fdc95ceeca3902
I20260808 07:00:39.127200 1805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:39.135766 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.124321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.24.181.126:43181)
0808 07:00:39.124511 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.124521 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.125362 (+ 841us) server_negotiation.cc:407] Connection header received
0808 07:00:39.125728 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.125738 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.125812 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.125963 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.126824 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.127955 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.128753 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.129163 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.129282 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.131697 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.131701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.131702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.131714 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.133944 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.134347 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.134350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.134351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.134460 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.134694 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.134695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.134696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.134984 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.135139 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.135372 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.135523 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260808 07:00:39.152197 1805 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:39.152752 1805 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:39.160045 1805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Generated new TSK 0
I20260808 07:00:39.160244 1805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
I20260808 07:00:39.191277 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:39.293886 1825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:39.294112 1825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:39.294154 1825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:39.295383 1825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:39.295450 1825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:39.295495 1825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:39.296970 1825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:39.297238 1825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:39.297488 1825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.302735 1833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.302726 1836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.302726 1834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.302959 1825 server_base.cc:1034] running on GCE node
I20260808 07:00:39.303164 1825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.303434 1825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.304576 1825 hybrid_clock.cc:648] HybridClock initialized: now 1786172439304555 us; error 41 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.307242 1825 init.cc:377] Logged in from keytab as oryx/127.24.181.65@KRBTEST.COM (short username oryx)
I20260808 07:00:39.307721 1825 webserver.cc:460] Webserver started at http://127.24.181.65:35525/ using document root <none> and password file <none>
I20260808 07:00:39.307966 1825 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.308036 1825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.308193 1825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:39.309227 1825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8b1a939cfc2541d589c3acd86e43c2dc"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "18ebfa46d5dff26047893f0d23a986c7"
server_key_iv: "09552c060fe8506c4a4bd1522d90287d"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.309587 1825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8b1a939cfc2541d589c3acd86e43c2dc"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "18ebfa46d5dff26047893f0d23a986c7"
server_key_iv: "09552c060fe8506c4a4bd1522d90287d"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.310904 1825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:00:39.311741 1846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.311923 1825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:39.312022 1825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8b1a939cfc2541d589c3acd86e43c2dc"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "18ebfa46d5dff26047893f0d23a986c7"
server_key_iv: "09552c060fe8506c4a4bd1522d90287d"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.312111 1825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.324164 1825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.326629 1825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:39.326787 1825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.327067 1825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:39.327391 1825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:39.327445 1825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.327481 1825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:39.327508 1825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.333562 1825 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36647
I20260808 07:00:39.333604 2012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36647 every 8 connection(s)
I20260808 07:00:39.333940 1825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:39.337294 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 1825
I20260808 07:00:39.337404 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:39.337635 25301 external_mini_cluster.cc:1428] Setting key 32c1d06cfff5d84a6da315270983aced
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:39.345264 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.334664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:37731)
0808 07:00:39.334815 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.334849 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.335418 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.335659 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.335662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.336223 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.336407 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.336411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.337302 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.337303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.338380 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.338383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.338838 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.338839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.339003 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.339548 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.339558 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.341732 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.343799 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.343806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.343807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.344029 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.344294 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.344295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.344295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.344438 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.344768 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.344774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.344883 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.344989 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.345106 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260808 07:00:39.345273 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.334679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:37731 (local address 127.24.181.126:43181)
0808 07:00:39.334727 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.334729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.334958 (+ 229us) server_negotiation.cc:407] Connection header received
0808 07:00:39.335521 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.335522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.335534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.335606 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.336502 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.337188 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.338467 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.338744 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.338817 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.341844 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.341845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.341846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.341854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.343640 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.344119 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.344120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.344121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.344192 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.344524 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.344525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.344526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.344678 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.344769 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.345043 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.345141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:39.346100 2013 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:39.346212 2013 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:39.346428 2013 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:39.347077 1734 ts_manager.cc:194] Registered new tserver with Master: 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65:36647)
WARNING: no policy specified for oryx/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:00:39.348044 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:37731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
I20260808 07:00:39.366308 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:39.456966 2019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:39.457199 2019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:39.457247 2019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:39.459342 2019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:39.459403 2019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:39.459451 2019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:39.461781 2019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:39.462060 2019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:39.462364 2019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.469080 2024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.469051 2025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.469187 2027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.469241 2019 server_base.cc:1034] running on GCE node
I20260808 07:00:39.469612 2019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.469875 2019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.471046 2019 hybrid_clock.cc:648] HybridClock initialized: now 1786172439471029 us; error 34 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.473759 2019 init.cc:377] Logged in from keytab as oryx/127.24.181.66@KRBTEST.COM (short username oryx)
I20260808 07:00:39.474412 2019 webserver.cc:460] Webserver started at http://127.24.181.66:44807/ using document root <none> and password file <none>
I20260808 07:00:39.474674 2019 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.474747 2019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.474917 2019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:39.476420 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "5fda071274ac444691618383b3ebeac2"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "5fd2701d1ca356bf05157f9cdc639fe9"
server_key_iv: "d00e30ba860a86d624672f0e5f496d77"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.476853 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "5fda071274ac444691618383b3ebeac2"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "5fd2701d1ca356bf05157f9cdc639fe9"
server_key_iv: "d00e30ba860a86d624672f0e5f496d77"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.478439 2019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:39.479493 2038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.479691 2019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:39.479772 2019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "5fda071274ac444691618383b3ebeac2"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "5fd2701d1ca356bf05157f9cdc639fe9"
server_key_iv: "d00e30ba860a86d624672f0e5f496d77"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.479846 2019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.531872 2019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.533946 2019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:39.534118 2019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.534396 2019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:39.534725 2019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:39.534785 2019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.534828 2019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:39.534852 2019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.540802 2019 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:45695
I20260808 07:00:39.540866 2211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:45695 every 8 connection(s)
I20260808 07:00:39.541169 2019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:39.542315 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 2019
I20260808 07:00:39.542434 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:39.542624 25301 external_mini_cluster.cc:1428] Setting key 75f85a3736897c952f3f55b6f649b5c3
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:39.551704 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.541990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:40057 (local address 127.24.181.126:43181)
0808 07:00:39.542086 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.542088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.542260 (+ 172us) server_negotiation.cc:407] Connection header received
0808 07:00:39.542982 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.542984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.542998 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.543058 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.544295 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.544921 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.545913 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.546127 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.546208 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.548845 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.548846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.548847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.548853 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.550220 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.550711 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.550712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.550712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.550764 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.551069 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.551070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.551070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.551172 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.551226 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.551579 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.551641 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.24.181.67@KRBTEST.COM; defaulting to no policy
I20260808 07:00:39.551973 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.541946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:40057)
0808 07:00:39.542137 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.542174 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.542807 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.543372 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.543377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.543986 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.544177 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.544182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.545019 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.545020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.545814 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.545816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.546216 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.546217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.546350 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.546910 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.546964 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.548710 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.550315 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.550323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.550325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.550633 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.550831 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.550831 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.550832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.550999 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.551242 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.551250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.551428 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.551563 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.551732 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260808 07:00:39.552572 2212 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:39.552671 2212 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:39.552811 2212 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:39.553231 1734 ts_manager.cc:194] Registered new tserver with Master: 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:39.553567 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:40057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
I20260808 07:00:39.570114 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:39.653081 2218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:39.653282 2218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:39.653306 2218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:39.654390 2218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:39.654425 2218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:39.654491 2218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:39.655951 2218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:39.656217 2218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:39.656401 2218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.661387 2227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.661355 2226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.661589 2229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.661561 2218 server_base.cc:1034] running on GCE node
I20260808 07:00:39.661829 2218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.662020 2218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.663156 2218 hybrid_clock.cc:648] HybridClock initialized: now 1786172439663143 us; error 30 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.665714 2218 init.cc:377] Logged in from keytab as oryx/127.24.181.67@KRBTEST.COM (short username oryx)
I20260808 07:00:39.666127 2218 webserver.cc:460] Webserver started at http://127.24.181.67:40059/ using document root <none> and password file <none>
I20260808 07:00:39.666357 2218 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.666425 2218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.666606 2218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:39.667794 2218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "4c43dc9b003298d64d029c4526ac8192"
server_key_iv: "5290fdf9bc188260327914717b3c5379"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.668164 2218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "4c43dc9b003298d64d029c4526ac8192"
server_key_iv: "5290fdf9bc188260327914717b3c5379"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.669513 2218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:00:39.670354 2236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.670548 2218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:39.670650 2218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "4c43dc9b003298d64d029c4526ac8192"
server_key_iv: "5290fdf9bc188260327914717b3c5379"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.670738 2218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.687981 2218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.690336 2218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:39.690497 2218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.690742 2218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:39.691104 2218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:39.691160 2218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.691197 2218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:39.691231 2218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.697227 2218 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:40399
I20260808 07:00:39.697276 2406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:40399 every 8 connection(s)
I20260808 07:00:39.697578 2218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:39.705271 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 2218
I20260808 07:00:39.705392 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:39.705587 25301 external_mini_cluster.cc:1428] Setting key 6669f6b12a18b2fc6728b66f0c86abb8
I20260808 07:00:39.707247 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.698300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:39641 (local address 127.24.181.126:43181)
0808 07:00:39.698362 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.698365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.698606 (+ 241us) server_negotiation.cc:407] Connection header received
0808 07:00:39.699263 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.699264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.699275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.699348 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.700257 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.700806 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.701842 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.702025 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.702082 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.704264 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.704266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.704266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.704273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.705716 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.706251 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.706253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.706253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.706328 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.706636 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.706637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.706637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.706762 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.706817 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.707115 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.707181 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:00:39.707398 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.698276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:39641)
0808 07:00:39.698462 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.698503 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.699078 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.699411 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.699414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.699976 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.700158 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.700162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.700891 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.700892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.701771 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.701774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.702103 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.702104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.702199 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.702707 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.702716 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.704180 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.705803 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.705810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.705811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.706156 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.706404 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.706405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.706406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.706557 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.706832 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.706838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.706991 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.707086 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.707205 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260808 07:00:39.707888 2407 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:39.707996 2407 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:39.708179 2407 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:39.708523 1734 ts_manager.cc:194] Registered new tserver with Master: aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:39.708910 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:39641
I20260808 07:00:39.717386 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.181.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.125.keytab.
Entry for principal oryx/127.24.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.125.keytab.
I20260808 07:00:39.745280 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:32777
--webserver_interface=127.24.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.125.keytab
--principal=oryx/127.24.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43181,127.24.181.125:32777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:39.826395 2415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:39.826588 2415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:39.826610 2415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:39.827943 2415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:39.827988 2415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:39.828003 2415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:39.828016 2415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:39.828029 2415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:39.829505 2415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43181,127.24.181.125:32777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.125.keytab
--principal=oryx/127.24.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:32777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
W20260808 07:00:39.829607 2415 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.181.126:43181,127.24.181.125:32777'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 07:00:39.829766 2415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:39.830006 2415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.834875 2423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.834883 2425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.834904 2422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.835085 2415 server_base.cc:1034] running on GCE node
I20260808 07:00:39.835362 2415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.835597 2415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.836761 2415 hybrid_clock.cc:648] HybridClock initialized: now 1786172439836737 us; error 39 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.839452 2415 init.cc:377] Logged in from keytab as oryx/127.24.181.125@KRBTEST.COM (short username oryx)
I20260808 07:00:39.839918 2415 webserver.cc:460] Webserver started at http://127.24.181.125:46511/ using document root <none> and password file <none>
I20260808 07:00:39.840154 2415 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.840199 2415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.840361 2415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:39.841369 2415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/instance:
uuid: "59d128f57b0c4ad8bd635f84a9b96134"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "032ae904d509ec5ef4c8ad3d73eb58aa"
server_key_iv: "57290a900b067e08891d4d1db758c739"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.841719 2415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal/instance:
uuid: "59d128f57b0c4ad8bd635f84a9b96134"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "032ae904d509ec5ef4c8ad3d73eb58aa"
server_key_iv: "57290a900b067e08891d4d1db758c739"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.843078 2415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:00:39.843796 2435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.843971 2415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:39.844070 2415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "59d128f57b0c4ad8bd635f84a9b96134"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "032ae904d509ec5ef4c8ad3d73eb58aa"
server_key_iv: "57290a900b067e08891d4d1db758c739"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.844182 2415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.855427 2415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.857688 2415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:39.857954 2415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.862464 2415 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.125:32777
I20260808 07:00:39.862497 2512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.125:32777 every 8 connection(s)
I20260808 07:00:39.862855 2415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
I20260808 07:00:39.863862 2513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:39.865475 2513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:00:39.870251 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 2415
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.125@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:39.875056 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.866194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.24.181.126:43181)
0808 07:00:39.866283 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.866286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.866540 (+ 254us) server_negotiation.cc:407] Connection header received
0808 07:00:39.867047 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.867049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.867060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.867121 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.867713 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.868197 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.869153 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.869339 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.869391 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.871962 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.871963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.871963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.871970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.873491 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.873956 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.873957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.873957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.874008 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.874306 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.874306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.874307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.874496 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.874567 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.874866 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.874993 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:00:39.875500 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.866137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43128)
0808 07:00:39.866424 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.866460 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.866899 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.867196 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.867199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.867453 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.867639 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.867642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.868273 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.868274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.869079 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.869080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.869414 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.869415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.869525 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.870016 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.870024 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.871875 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.873583 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.873590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.873592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.873878 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.874071 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.874071 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.874072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.874227 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.874555 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.874560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.874700 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.874842 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.875343 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":151,"threads_started":1}
I20260808 07:00:39.876121 2513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.125@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
I20260808 07:00:39.880316 2517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.870957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.24.181.125:32777)
0808 07:00:39.871123 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.871127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.871141 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:00:39.871211 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.871215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.871232 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.871305 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.872065 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.873134 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.874157 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.874593 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.874650 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.877282 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.877291 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.877291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.877299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.878839 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.879321 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.879323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.879324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.879379 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.879612 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.879613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.879613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.879812 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.879959 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.880044 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.880117 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260808 07:00:39.883884 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.876366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47382)
0808 07:00:39.876412 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.876422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.876456 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.876719 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.876719 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.876862 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.876926 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.876928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.877738 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.877738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.878198 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.878199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.878605 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.878605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.878673 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.879088 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.879093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.880561 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.882787 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.882788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.882789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.883037 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.883273 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.883274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.883275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.883331 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.883573 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.883574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.883610 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.883679 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.883770 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:00:39.883965 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.876462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.24.181.125:32777)
0808 07:00:39.876613 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.876614 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.876619 (+ 5us) server_negotiation.cc:407] Connection header received
0808 07:00:39.876633 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.876633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.876640 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.876716 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.876994 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.877452 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.878299 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.878518 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.878604 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.880679 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.880681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.880681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.880687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.882683 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.883146 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.883147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.883147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.883207 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.883403 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.883404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.883404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.883506 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.883581 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.883704 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.883821 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260808 07:00:39.885239 2513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Bootstrap starting.
I20260808 07:00:39.885808 2513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:39.886036 2513 log.cc:824] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:39.886744 2513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: No bootstrap required, opened a new log
I20260808 07:00:39.888164 2513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } }
I20260808 07:00:39.888285 2513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:39.888307 2513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d128f57b0c4ad8bd635f84a9b96134, State: Initialized, Role: FOLLOWER
I20260808 07:00:39.888435 2513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } }
I20260808 07:00:39.888682 2520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } } }
I20260808 07:00:39.888744 2513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:39.888761 2520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:00:39.890416 2533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:00:39.890481 2533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:00:39.892040 2415 master_runner.cc:428] Detected that this master 59d128f57b0c4ad8bd635f84a9b96134 is joining an existing cluster
I20260808 07:00:39.892092 2415 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.181.125:32777
I20260808 07:00:39.899912 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.893381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47392 (local address 127.24.181.125:32777)
0808 07:00:39.893419 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.893421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.893526 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:00:39.893618 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.893619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.893628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.893671 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.894017 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.894492 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.895333 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.895546 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.895631 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.896949 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.896950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.896950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.896957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.898529 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.899020 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.899021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.899021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.899086 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.899306 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.899307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.899308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.899422 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.899493 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.899652 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.899788 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:00:39.900169 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.893350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47392)
0808 07:00:39.893469 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.893479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.893547 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.893685 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.893686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.893878 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.893930 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.893932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.894573 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.894574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.895239 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.895241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.895656 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.895658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.895716 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.896371 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.896375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.896855 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.898615 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.898615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.898616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.898877 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.899170 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.899171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.899171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.899242 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.899495 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.899496 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.899535 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.899629 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.899850 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":117,"thread_start_us":67,"threads_started":1}
I20260808 07:00:39.901206 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.893205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.24.181.126:43181)
0808 07:00:39.893255 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.893257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.893353 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:00:39.893472 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.893474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.893489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.893552 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.894027 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.894781 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.895850 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.896072 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.896152 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.897119 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.897121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.897121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.897130 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.899449 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.899903 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.899904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.899904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.900379 (+ 475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.900609 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.900610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.900611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.900759 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.900808 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.901073 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.901145 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:00:39.901237 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.893102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43132)
0808 07:00:39.893280 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.893292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.893383 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.893565 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.893566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.893845 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.893924 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.893926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.894896 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.894897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.895750 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.895753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.896148 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.896149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.896238 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.896627 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.896632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.897024 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.899542 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.899543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.899544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.899802 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.900471 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.900471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.900472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.900538 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.900872 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.900872 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.900927 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.900999 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.901118 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.126@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
I20260808 07:00:39.911250 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.903850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47396)
0808 07:00:39.903986 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.904017 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.904079 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.904221 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.904223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.904466 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.904513 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.904515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.905380 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.905381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.905933 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.905934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.906391 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.906396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.906456 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.906902 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.906906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.908462 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.910125 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.910132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.910134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.910348 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.910563 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.910564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.910564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.910620 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.910891 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.910898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.911003 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.911074 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.911165 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260808 07:00:39.911300 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.903887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.24.181.125:32777)
0808 07:00:39.903922 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.903924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.904085 (+ 161us) server_negotiation.cc:407] Connection header received
0808 07:00:39.904139 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.904140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.904150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.904218 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.904584 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.905299 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.906011 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.906311 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.906382 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.908541 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.908543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.908543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.908550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.910012 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.910427 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.910428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.910429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.910492 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.910685 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.910686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.910686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.910811 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.910894 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.911134 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.911211 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:00:39.912168 1734 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.181.125:32777: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134"
member_type: NON_VOTER
last_known_addr {
host: "127.24.181.125"
port: 32777
}
attrs {
promote: true
}
}
dest_uuid: "ec45ae6459b4465bb4f3616db3eb32b3"
cas_config_opid_index: -1
I20260808 07:00:39.912549 1734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } }
I20260808 07:00:39.913074 2548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 59d128f57b0c4ad8bd635f84a9b96134 (127.24.181.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } }
I20260808 07:00:39.913347 2474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:39.914068 2548 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.181.125:32777
I20260808 07:00:39.914340 2474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 07:00:39.914350 2415 master.cc:537] Master@127.24.181.125:32777 shutting down...
I20260808 07:00:39.914340 2549 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } }
I20260808 07:00:39.914422 2549 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:39.914975 2549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:00:39.915795 1719 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 07:00:39.916683 1719 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 -> Peer 59d128f57b0c4ad8bd635f84a9b96134 (127.24.181.125:32777): Couldn't send request to peer 59d128f57b0c4ad8bd635f84a9b96134. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:00:39.917477 2415 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:39.917565 2415 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:39.917616 2415 tablet_replica.cc:330] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: stopping tablet replica
I20260808 07:00:39.930346 2415 master.cc:552] Master@127.24.181.125:32777 shutdown complete.
I20260808 07:00:39.930400 2415 master_runner.cc:305] Clearing existing system tablet
I20260808 07:00:39.930543 2415 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.930593 2415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.931432 2415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:39.931998 2555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.932143 2415 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.932209 2415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:39.932236 2415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "59d128f57b0c4ad8bd635f84a9b96134"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "032ae904d509ec5ef4c8ad3d73eb58aa"
server_key_iv: "57290a900b067e08891d4d1db758c739"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.933051 2415 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:39.934509 2415 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:00:39.934585 2415 log.cc:1196] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 07:00:39.934783 2415 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Deleting consensus metadata
I20260808 07:00:39.934978 2415 master_runner.cc:315] Copying system tablet from 127.24.181.126:43181
I20260808 07:00:39.935585 2415 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: tablet copy: Beginning tablet copy session from remote peer at address 127.24.181.126:43181
I20260808 07:00:39.942023 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.935940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.24.181.126:43181)
0808 07:00:39.935992 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:39.935994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:39.936071 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:00:39.936129 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:39.936130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:39.936139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:39.936183 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:39.936560 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.937055 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.938107 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.938290 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.938345 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.939395 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:39.939396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:39.939397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:39.939403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:39.940896 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.941295 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.941296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.941296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.941351 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:39.941555 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:39.941555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:39.941555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:39.941653 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:39.941730 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:39.941900 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:00:39.941962 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:00:39.942090 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:39.935853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43142)
0808 07:00:39.935989 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:39.936001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:39.936068 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:39.936198 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:39.936199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:39.936421 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:39.936491 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.936493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.937136 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.937136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.938020 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:39.938022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:39.938365 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:39.938366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:39.938440 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:39.938890 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:39.938896 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:39.939299 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:39.940994 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.940996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.940996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.941217 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.941419 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:39.941420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:39.941421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:39.941491 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:39.941718 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:39.941719 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:39.941770 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:00:39.941875 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:00:39.941977 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":94,"threads_started":1}
I20260808 07:00:39.942736 1760 tablet_copy_service.cc:140] P ec45ae6459b4465bb4f3616db3eb32b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 59d128f57b0c4ad8bd635f84a9b96134 ({username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:43142)
I20260808 07:00:39.942859 1760 tablet_copy_service.cc:161] P ec45ae6459b4465bb4f3616db3eb32b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 59d128f57b0c4ad8bd635f84a9b96134 at {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:43142: session id = 59d128f57b0c4ad8bd635f84a9b96134-00000000000000000000000000000000
I20260808 07:00:39.943648 1760 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:00:39.944327 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:00:39.945648 2415 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: tablet copy: Starting download of 0 data blocks...
I20260808 07:00:39.945804 2415 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: tablet copy: Starting download of 1 WAL segments...
I20260808 07:00:39.946899 2415 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:00:39.947870 1760 tablet_copy_service.cc:342] P ec45ae6459b4465bb4f3616db3eb32b3: Request end of tablet copy session 59d128f57b0c4ad8bd635f84a9b96134-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:43142
I20260808 07:00:39.947945 1760 tablet_copy_service.cc:433] P ec45ae6459b4465bb4f3616db3eb32b3: ending tablet copy session 59d128f57b0c4ad8bd635f84a9b96134-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 59d128f57b0c4ad8bd635f84a9b96134
I20260808 07:00:39.950421 2415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:39.951668 2569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.951679 2572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:39.951746 2570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:39.955073 2415 server_base.cc:1034] running on GCE node
I20260808 07:00:39.955178 2415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:39.955317 2415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:39.956458 2415 hybrid_clock.cc:648] HybridClock initialized: now 1786172439956453 us; error 36 us; skew 500 ppm
Aug 08 07:00:39 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:39.957864 2415 init.cc:377] Logged in from keytab as oryx/127.24.181.125@KRBTEST.COM (short username oryx)
I20260808 07:00:39.958169 2415 webserver.cc:460] Webserver started at http://127.24.181.125:46129/ using document root <none> and password file <none>
I20260808 07:00:39.958318 2415 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:39.958369 2415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:39.959013 2415 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:39.959580 2582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.959743 2415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:39.959802 2415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "59d128f57b0c4ad8bd635f84a9b96134"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "032ae904d509ec5ef4c8ad3d73eb58aa"
server_key_iv: "57290a900b067e08891d4d1db758c739"
server_key_version: "encryptionkey@0"
I20260808 07:00:39.959860 2415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:39.984743 2415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:39.986598 2415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:39.989863 2415 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.125:32777
I20260808 07:00:39.989884 2663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.125:32777 every 8 connection(s)
I20260808 07:00:39.990211 2415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
I20260808 07:00:39.991154 2664 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:00:39.991709 2664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Bootstrap starting.
I20260808 07:00:39.994781 2664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:00:39.995090 2664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Bootstrap complete.
I20260808 07:00:39.995314 2664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } }
I20260808 07:00:39.995409 2664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59d128f57b0c4ad8bd635f84a9b96134, State: Initialized, Role: LEARNER
I20260808 07:00:39.995451 2664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } }
I20260808 07:00:39.995626 2664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:39.995645 2667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } }
I20260808 07:00:39.995812 2667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:00:39.997226 2681 catalog_manager.cc:1259] Loaded cluster ID: 062349e0c0964164a4fdc95ceeca3902
I20260808 07:00:39.997599 2681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: loading cluster ID for follower catalog manager: success
I20260808 07:00:39.998318 2681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: acquiring CA information for follower catalog manager: success
I20260808 07:00:39.998591 2681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:00:40.039795 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.032401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.24.181.125:32777)
0808 07:00:40.032531 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.032533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.032541 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:40.032566 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.032566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.032576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.032655 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.033483 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.034175 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.035242 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.035432 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.035497 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.036909 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.036910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.036910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.036917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.038475 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.038894 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.038896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.038896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.039012 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.039209 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.039210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.039210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.039319 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.039396 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.039527 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.039595 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.24.181.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.124.keytab.
Entry for principal oryx/127.24.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.124.keytab.
I20260808 07:00:40.067700 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:45385
--webserver_interface=127.24.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.124.keytab
--principal=oryx/127.24.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:40.151607 2688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:40.151826 2688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:40.151850 2688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:40.152998 2688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:40.153043 2688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:40.153057 2688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:40.153070 2688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:40.153124 2688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:40.154605 2688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.124.keytab
--principal=oryx/127.24.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:45385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:40.154870 2688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:40.155133 2688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:40.160280 2698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.160346 2688 server_base.cc:1034] running on GCE node
W20260808 07:00:40.160290 2694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.160300 2695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.160754 2688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:40.161087 2688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:40.162248 2688 hybrid_clock.cc:648] HybridClock initialized: now 1786172440162238 us; error 28 us; skew 500 ppm
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:40.164978 2688 init.cc:377] Logged in from keytab as oryx/127.24.181.124@KRBTEST.COM (short username oryx)
I20260808 07:00:40.165511 2688 webserver.cc:460] Webserver started at http://127.24.181.124:41673/ using document root <none> and password file <none>
I20260808 07:00:40.165753 2688 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.165823 2688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.165982 2688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:40.167174 2688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/instance:
uuid: "9c2c50c360664922b08f68c464f1402f"
format_stamp: "Formatted at 2026-08-08 07:00:40 on dist-test-slave-nfb5"
server_key: "ce322e66a60be66ad0b0552252efe6ab"
server_key_iv: "3ca1fe58c7a7a373711bf3ec3212cf88"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.167560 2688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal/instance:
uuid: "9c2c50c360664922b08f68c464f1402f"
format_stamp: "Formatted at 2026-08-08 07:00:40 on dist-test-slave-nfb5"
server_key: "ce322e66a60be66ad0b0552252efe6ab"
server_key_iv: "3ca1fe58c7a7a373711bf3ec3212cf88"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.168926 2688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:40.169703 2708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.169884 2688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:40.169992 2688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "9c2c50c360664922b08f68c464f1402f"
format_stamp: "Formatted at 2026-08-08 07:00:40 on dist-test-slave-nfb5"
server_key: "ce322e66a60be66ad0b0552252efe6ab"
server_key_iv: "3ca1fe58c7a7a373711bf3ec3212cf88"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.170068 2688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.185913 2688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:40.188035 2688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:40.188216 2688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:40.192579 2688 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.124:45385
I20260808 07:00:40.192615 2778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.124:45385 every 8 connection(s)
I20260808 07:00:40.193011 2688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
I20260808 07:00:40.193291 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 2688
I20260808 07:00:40.194358 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:40.196673 2782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.124@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:40.205047 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.194123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.24.181.124:45385)
0808 07:00:40.194329 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.194338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.194373 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:00:40.195050 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.195059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.195124 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.195281 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.195996 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.197452 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.198222 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.198673 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.198754 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.201615 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.201623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.201624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.201630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.203522 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.203915 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.203919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.203919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.204011 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.204230 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.204231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.204232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.204461 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.204600 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.204721 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.204863 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260808 07:00:40.205399 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.197336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43150)
0808 07:00:40.197541 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.197580 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.197641 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.197803 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.197806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.198101 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.198216 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.198218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.199058 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.199060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.199749 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.199752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.200316 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.200317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.200371 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.200849 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.200857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.202535 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.204237 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.204243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.204245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.204444 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.204674 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.204675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.204675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.204740 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.205001 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.205008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.205057 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.205167 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.205283 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260808 07:00:40.205575 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.197467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.24.181.126:43181)
0808 07:00:40.197541 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.197542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.197633 (+ 91us) server_negotiation.cc:407] Connection header received
0808 07:00:40.197723 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.197724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.197736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.197787 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.198309 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.198881 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.199850 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.200195 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.200274 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.202655 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.202656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.202656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.202664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.204140 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.204535 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.204536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.204537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.204600 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.204808 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.204809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.204810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.204948 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.205009 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.205386 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.205471 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:00:40.205997 2782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.124@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
I20260808 07:00:40.212260 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.206239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47426)
0808 07:00:40.206291 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.206302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.206343 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.206501 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.206502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.206638 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.206682 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.206684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.207431 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.207432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.207935 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.207937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.208311 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.208312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.208376 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.208686 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.208690 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.209852 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.211447 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.211448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.211448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.211610 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.211791 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.211791 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.211791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.211842 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.212048 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.212049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.212081 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.212130 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.212197 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:00:40.212271 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.206332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.24.181.125:32777)
0808 07:00:40.206378 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.206380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.206388 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:40.206416 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.206417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.206430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.206500 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.206785 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.207353 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.208020 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.208241 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.208314 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.209929 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.209930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.209930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.209937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.211373 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.211676 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.211676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.211677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.211726 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.211894 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.211895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.211895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.211994 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.212044 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.212147 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.212210 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:40.212742 2782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.124@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
I20260808 07:00:40.218453 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.213006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47854)
0808 07:00:40.213054 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.213067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.213146 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.213268 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.213269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.213376 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.213402 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.213403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.213968 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.213968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.214411 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.214412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.214741 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.214742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.214776 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.215144 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.215148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.216200 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.217721 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.217722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.217722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.217868 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.218034 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.218034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.218034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.218081 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.218247 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.218248 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.218278 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.218326 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.218395 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:00:40.218514 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.213059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47854 (local address 127.24.181.124:45385)
0808 07:00:40.213116 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.213118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.213133 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:00:40.213214 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.213214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.213221 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.213265 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.213459 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.213901 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.214490 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.214680 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.214727 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.216277 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.216278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.216278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.216284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.217647 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.217930 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.217930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.217931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.217978 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.218119 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.218119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.218119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.218200 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.218242 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.218343 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.218440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":38}
I20260808 07:00:40.219856 2782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Bootstrap starting.
I20260808 07:00:40.220709 2782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:40.220997 2782 log.cc:824] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:40.221748 2782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: No bootstrap required, opened a new log
I20260808 07:00:40.223233 2782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } }
I20260808 07:00:40.223380 2782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:40.223426 2782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2c50c360664922b08f68c464f1402f, State: Initialized, Role: FOLLOWER
I20260808 07:00:40.223593 2782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } }
I20260808 07:00:40.224015 2782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:40.224035 2787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } } }
I20260808 07:00:40.224197 2787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:00:40.225833 2801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:00:40.225898 2801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:00:40.228220 2688 master_runner.cc:428] Detected that this master 9c2c50c360664922b08f68c464f1402f is joining an existing cluster
I20260808 07:00:40.228276 2688 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.181.124:45385
I20260808 07:00:40.238014 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.24.181.125:32777)
0808 07:00:40.229784 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.229786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.231247 (+ 1461us) server_negotiation.cc:407] Connection header received
0808 07:00:40.231366 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.231367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.231380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.231410 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.231801 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232829 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.234048 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.234308 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.234380 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.235307 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.235308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.235309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.235316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.236859 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.237213 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.237215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.237215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.237280 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.237492 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.237493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.237494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.237606 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.237679 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.237852 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.237925 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:40.238023 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47428)
0808 07:00:40.231196 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.231204 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.231280 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.231473 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.231473 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.231646 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.231718 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.231719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232921 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.232922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.233958 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.233960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.234459 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.234460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.234527 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.234897 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.234901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.235217 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.236938 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.236939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.236939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.237112 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.237361 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.237361 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.237362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.237420 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.237669 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.237670 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.237718 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.237816 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.237918 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495,"thread_start_us":73,"threads_started":1}
I20260808 07:00:40.256695 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 1825
I20260808 07:00:40.281977 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.24.181.124:45385)
0808 07:00:40.229661 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.229663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.229902 (+ 239us) server_negotiation.cc:407] Connection header received
0808 07:00:40.229959 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.229960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.229967 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.230008 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.230385 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.230831 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.231919 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232194 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.232256 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.233400 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.233401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.233402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.233408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.279653 (+ 46245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.280334 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.280335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.280335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.280393 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.280973 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.280975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.280975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.281433 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.281484 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.281711 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.281812 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:00:40.282305 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47856)
0808 07:00:40.229823 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.229836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.229900 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.230029 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.230030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.230243 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.230315 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.230317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.230932 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.230934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.231823 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.231826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232334 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.232335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.232391 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.232835 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.232840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.233297 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.279903 (+ 46606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.279905 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.279905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.280236 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.280456 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.280457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.280457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.280881 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.281528 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.281529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.281597 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.282090 (+ 493us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.282214 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":98,"threads_started":1}
I20260808 07:00:40.283777 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43160)
0808 07:00:40.231012 (+ 1584us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.231021 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.231101 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.231289 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.231290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.231518 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.231580 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.231582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232549 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.232550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.233437 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.233439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.233949 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.233950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.234018 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.234454 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.234458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.280080 (+ 45622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.282196 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.282197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.282197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.282438 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.282767 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.282768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.282768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.282888 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.283383 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.283384 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.283449 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.283540 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.283666 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1568,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260808 07:00:40.283811 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.229524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.24.181.126:43181)
0808 07:00:40.229600 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.229602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.231094 (+ 1492us) server_negotiation.cc:407] Connection header received
0808 07:00:40.231164 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.231165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.231178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.231231 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.231667 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.232450 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.233528 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.233829 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.233873 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.280234 (+ 46361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.280236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.280237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.280249 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.282056 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.282628 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.282629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.282629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.282692 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.283024 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.283025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.283025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.283237 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.283316 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.283599 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.283696 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:00:40.284767 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:36647
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:43181
--webserver_port=35525
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172439, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.126@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
I20260808 07:00:40.294301 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.286140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.24.181.124:45385)
0808 07:00:40.286172 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.286174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.286181 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:00:40.286224 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.286224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.286233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.286281 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.286652 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.287456 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.288555 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.288873 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.288955 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.291246 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.291248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.291248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.291256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.292980 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.293437 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.293438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.293439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.293500 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.293732 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.293733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.293733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.293862 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.293931 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.294117 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.294197 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:00:40.294350 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.285941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47872)
0808 07:00:40.286024 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.286034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.286070 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.286331 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.286332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.286518 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.286566 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.286568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.287563 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.287564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.288430 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.288433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.288964 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.288965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.289044 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.289420 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.289426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.291135 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.293076 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.293077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.293078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.293352 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.293581 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.293581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.293582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.293659 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.293940 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.293941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.293997 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.294080 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.294208 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 07:00:40.295553 1734 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.181.124:45385: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9c2c50c360664922b08f68c464f1402f"
member_type: NON_VOTER
last_known_addr {
host: "127.24.181.124"
port: 45385
}
attrs {
promote: true
}
}
dest_uuid: "ec45ae6459b4465bb4f3616db3eb32b3"
cas_config_opid_index: 5
I20260808 07:00:40.295876 1734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } }
I20260808 07:00:40.296494 2548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9c2c50c360664922b08f68c464f1402f (127.24.181.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.297492 2548 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.181.124:45385
I20260808 07:00:40.297556 2548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.297632 2548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.297896 2688 master.cc:537] Master@127.24.181.124:45385 shutting down...
I20260808 07:00:40.298002 2746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:40.299086 2746 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:00:40.300103 2549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:00:40.300798 1714 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 -> Peer 9c2c50c360664922b08f68c464f1402f (127.24.181.124:45385): Couldn't send request to peer 9c2c50c360664922b08f68c464f1402f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:00:40.302685 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.296392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47438)
0808 07:00:40.298234 (+ 1842us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.298245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.298767 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.299050 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.299051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.299052 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.299088 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.299090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.300030 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.300031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.301515 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.301519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.302437 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.302438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.302504 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.302524 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.302582 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.302620 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1823}
I20260808 07:00:40.303056 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.296508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.24.181.125:32777)
0808 07:00:40.296559 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.296561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.298736 (+ 2175us) server_negotiation.cc:407] Connection header received
0808 07:00:40.298851 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.298852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.298878 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.298983 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.299157 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.299938 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.301627 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.302264 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.302314 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.302378 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.302949 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.302996 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:40.303354 2688 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:40.303447 2688 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:40.303496 2688 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: stopping tablet replica
I20260808 07:00:40.303647 2624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 LEARNER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:00:40.304070 2549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 07:00:40.304677 2624 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9c2c50c360664922b08f68c464f1402f (127.24.181.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.305996 2667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec45ae6459b4465bb4f3616db3eb32b3. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.306108 2667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:00:40.306157 2667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: NON_VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: true } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.306223 2667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:00:40.306782 2688 master.cc:552] Master@127.24.181.124:45385 shutdown complete.
I20260808 07:00:40.306830 2688 master_runner.cc:305] Clearing existing system tablet
I20260808 07:00:40.306996 2688 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.307047 2688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.307741 2549 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: attempting to promote NON_VOTER 59d128f57b0c4ad8bd635f84a9b96134 to VOTER
I20260808 07:00:40.307934 2549 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } }
I20260808 07:00:40.308390 2624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 LEARNER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:00:40.308430 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.308337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47876)
0808 07:00:40.308372 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.308406 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:00:40.308673 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.308610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47878)
0808 07:00:40.308639 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.308656 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 07:00:40.308729 2548 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 07:00:40.308880 1714 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 -> Peer 9c2c50c360664922b08f68c464f1402f (127.24.181.124:45385): Couldn't send request to peer 9c2c50c360664922b08f68c464f1402f. Status: Network error: Client connection negotiation failed: client connection to 127.24.181.124:45385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:00:40.308962 2688 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:40.309334 2548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 59d128f57b0c4ad8bd635f84a9b96134 (127.24.181.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.310204 2548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.310273 2548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.310268 2819 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.310442 2819 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.310663 2624 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 59d128f57b0c4ad8bd635f84a9b96134 (127.24.181.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.311535 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.311640 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:00:40.311888 2820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.312032 2688 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.312099 2688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260808 07:00:40.312124 2688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "9c2c50c360664922b08f68c464f1402f"
format_stamp: "Formatted at 2026-08-08 07:00:40 on dist-test-slave-nfb5"
server_key: "ce322e66a60be66ad0b0552252efe6ab"
server_key_iv: "3ca1fe58c7a7a373711bf3ec3212cf88"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.312866 2688 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:40.314209 2688 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:00:40.314297 2688 log.cc:1196] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 07:00:40.314502 2688 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Deleting consensus metadata
I20260808 07:00:40.314625 2688 master_runner.cc:315] Copying system tablet from 127.24.181.126:43181
I20260808 07:00:40.315264 2688 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.181.126:43181
I20260808 07:00:40.322395 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.315538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43162)
0808 07:00:40.315668 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.315678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.315732 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.315868 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.315869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.316096 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.316144 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.316146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.317044 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.317045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.317847 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.317850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.318338 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.318339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.318408 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.318974 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.318980 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.319374 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.321287 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.321289 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.321289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.321508 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.321731 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.321732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.321732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.321799 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.322083 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.322084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.322130 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.322206 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.322303 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 07:00:40.322415 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.315595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.24.181.126:43181)
0808 07:00:40.315646 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.315648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.315721 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:00:40.315780 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.315780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.315791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.315861 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.316240 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.316947 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.317947 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.318250 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.318341 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.319471 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.319472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.319473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.319480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.321187 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.321590 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.321591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.321591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.321656 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.321878 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.321879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.321880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.322013 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.322083 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.322252 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.322332 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:00:40.323174 1760 tablet_copy_service.cc:140] P ec45ae6459b4465bb4f3616db3eb32b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9c2c50c360664922b08f68c464f1402f ({username='oryx', principal='oryx/127.24.181.124@KRBTEST.COM'} at 127.0.0.1:43162)
I20260808 07:00:40.323282 1760 tablet_copy_service.cc:161] P ec45ae6459b4465bb4f3616db3eb32b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9c2c50c360664922b08f68c464f1402f at {username='oryx', principal='oryx/127.24.181.124@KRBTEST.COM'} at 127.0.0.1:43162: session id = 9c2c50c360664922b08f68c464f1402f-00000000000000000000000000000000
I20260808 07:00:40.323885 1760 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:00:40.324378 2688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:00:40.325659 2688 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: tablet copy: Starting download of 0 data blocks...
I20260808 07:00:40.325817 2688 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: tablet copy: Starting download of 1 WAL segments...
I20260808 07:00:40.326558 2688 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:00:40.327492 1760 tablet_copy_service.cc:342] P ec45ae6459b4465bb4f3616db3eb32b3: Request end of tablet copy session 9c2c50c360664922b08f68c464f1402f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.181.124@KRBTEST.COM'} at 127.0.0.1:43162
I20260808 07:00:40.327548 1760 tablet_copy_service.cc:433] P ec45ae6459b4465bb4f3616db3eb32b3: ending tablet copy session 9c2c50c360664922b08f68c464f1402f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9c2c50c360664922b08f68c464f1402f
I20260808 07:00:40.330017 2688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:40.331254 2836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.331377 2831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.331545 2832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.331907 2688 server_base.cc:1034] running on GCE node
I20260808 07:00:40.331992 2688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:40.332116 2688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:40.333240 2688 hybrid_clock.cc:648] HybridClock initialized: now 1786172440333229 us; error 27 us; skew 500 ppm
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:40.334779 2688 init.cc:377] Logged in from keytab as oryx/127.24.181.124@KRBTEST.COM (short username oryx)
I20260808 07:00:40.335047 2688 webserver.cc:460] Webserver started at http://127.24.181.124:45725/ using document root <none> and password file <none>
I20260808 07:00:40.335153 2688 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.335189 2688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.335897 2688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:40.336517 2844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.336681 2688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.336730 2688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "9c2c50c360664922b08f68c464f1402f"
format_stamp: "Formatted at 2026-08-08 07:00:40 on dist-test-slave-nfb5"
server_key: "ce322e66a60be66ad0b0552252efe6ab"
server_key_iv: "3ca1fe58c7a7a373711bf3ec3212cf88"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.336776 2688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.357403 2688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:40.359441 2688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:40.363107 2688 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.124:45385
I20260808 07:00:40.363485 2920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.124:45385 every 8 connection(s)
I20260808 07:00:40.363574 2688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
I20260808 07:00:40.367550 2921 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:00:40.368026 2921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Bootstrap starting.
I20260808 07:00:40.370314 2921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:00:40.370548 2921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: Bootstrap complete.
I20260808 07:00:40.370780 2921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } }
I20260808 07:00:40.370877 2921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c2c50c360664922b08f68c464f1402f, State: Initialized, Role: LEARNER
I20260808 07:00:40.370947 2921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } }
I20260808 07:00:40.371143 2921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:40.371129 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.371371 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: LEARNER
I20260808 07:00:40.372821 2940 catalog_manager.cc:1259] Loaded cluster ID: 062349e0c0964164a4fdc95ceeca3902
I20260808 07:00:40.372910 2940 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: loading cluster ID for follower catalog manager: success
I20260808 07:00:40.373982 2940 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: acquiring CA information for follower catalog manager: success
I20260808 07:00:40.374321 2940 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 07:00:40.403374 2814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:40.403647 2814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:40.403690 2814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:40.405689 2814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:40.405750 2814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:40.405792 2814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:40.407799 2814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:36647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=35525
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:40.408026 2814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:40.408281 2814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:40.413683 2948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.413758 2947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.413748 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.413902 2814 server_base.cc:1034] running on GCE node
I20260808 07:00:40.414098 2814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:40.414321 2814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:40.415478 2814 hybrid_clock.cc:648] HybridClock initialized: now 1786172440415465 us; error 30 us; skew 500 ppm
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:40.418007 2814 init.cc:377] Logged in from keytab as oryx/127.24.181.65@KRBTEST.COM (short username oryx)
I20260808 07:00:40.418466 2814 webserver.cc:460] Webserver started at http://127.24.181.65:35525/ using document root <none> and password file <none>
I20260808 07:00:40.418648 2814 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.418702 2814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.420047 2814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:40.420598 2960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.420823 2814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:40.420912 2814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8b1a939cfc2541d589c3acd86e43c2dc"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "18ebfa46d5dff26047893f0d23a986c7"
server_key_iv: "09552c060fe8506c4a4bd1522d90287d"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.421226 2814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.428478 2814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:40.430182 2814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:40.430289 2814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:40.430457 2814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:40.430692 2814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:40.430732 2814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.430752 2814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:40.430765 2814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.436926 2814 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36647
I20260808 07:00:40.437026 3119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36647 every 8 connection(s)
I20260808 07:00:40.437345 2814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:40.443003 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 2814
I20260808 07:00:40.443186 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 2019
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:40.450264 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:41953 (local address 127.24.181.126:43181)
0808 07:00:40.438140 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.438142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.438396 (+ 254us) server_negotiation.cc:407] Connection header received
0808 07:00:40.439037 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.439039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.439054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.439113 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.439941 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.440724 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.441643 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.441866 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.441948 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.447269 (+ 5321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.447271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.447271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.447279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.448980 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449372 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.449373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.449374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.449427 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449646 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.449647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.449647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.449784 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.449847 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.450088 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.450174 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:00:40.450345 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:35903 (local address 127.24.181.125:32777)
0808 07:00:40.438627 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.438629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.438779 (+ 150us) server_negotiation.cc:407] Connection header received
0808 07:00:40.439075 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.439077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.439088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.439118 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.439944 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.440434 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.441662 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.442003 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.442038 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.446547 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.446549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.446549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.446564 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.448616 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449137 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.449139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.449139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.449209 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449521 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.449522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.449522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.449694 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.449769 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.450111 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.450207 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:40.450642 3125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:41953)
0808 07:00:40.438275 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.438308 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.438933 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.439173 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.439176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.439778 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.439826 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.439827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.440820 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.440821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.441546 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.441548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.442014 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.442015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.442162 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.442773 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.442781 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.447125 (+ 4344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.449068 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.449070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.449304 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.449500 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449500 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.449501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.449572 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.449849 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.449850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.449952 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.450065 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.450228 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260808 07:00:40.450779 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.24.181.65:35903)
0808 07:00:40.438696 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.438708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.438979 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.439191 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.439192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.439628 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.439816 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.439826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.440545 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.440546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.441550 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.441553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.442102 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.442103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.442184 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.442796 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.442802 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.446320 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.448705 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.448714 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.448716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.449039 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.449279 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.449279 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.449280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.449428 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.449769 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.449777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.449954 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.450431 (+ 477us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.450552 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":77,"threads_started":1}
I20260808 07:00:40.451359 3121 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:40.451464 3121 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.451656 3121 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:40.452143 1734 ts_manager.cc:194] Re-registered known tserver with Master: 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65:36647)
I20260808 07:00:40.452514 3122 heartbeater.cc:347] Connected to a master server at 127.24.181.125:32777
I20260808 07:00:40.452555 3122 heartbeater.cc:464] Registering TS with master...
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
I20260808 07:00:40.452610 3122 heartbeater.cc:511] Master 127.24.181.125:32777 requested a full tablet report, sending...
I20260808 07:00:40.452672 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:41953
I20260808 07:00:40.453034 2612 ts_manager.cc:194] Registered new tserver with Master: 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65:36647)
I20260808 07:00:40.453040 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:45695
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:43181
--webserver_port=44807
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:00:40.456066 3128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.24.181.65:50027)
0808 07:00:40.438558 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.438568 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.438940 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.439294 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.439294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.439631 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.439818 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.439824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.441097 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.441098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.441850 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.441852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.442393 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.442394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.442455 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.451391 (+ 8936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.451397 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.452951 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.454904 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.454905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.454906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.455098 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.455314 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.455315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.455316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.455375 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.455673 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.455674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.455710 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.455765 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.455852 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":2,"spinlock_wait_cycles":7552,"thread_start_us":110,"threads_started":1}
I20260808 07:00:40.456074 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.438408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:50027 (local address 127.24.181.124:45385)
0808 07:00:40.438618 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.438620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.438631 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:40.439177 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.439179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.439191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.439255 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.439937 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.441000 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.441933 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.442261 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.442337 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.453057 (+ 10720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.453059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.453059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.453069 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.454814 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.455177 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.455178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.455179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.455243 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.455455 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.455456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.455456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.455602 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.455674 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.455823 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.455926 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260808 07:00:40.456573 3120 heartbeater.cc:347] Connected to a master server at 127.24.181.124:45385
I20260808 07:00:40.456647 3120 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.456725 3120 heartbeater.cc:511] Master 127.24.181.124:45385 requested a full tablet report, sending...
I20260808 07:00:40.457062 2870 ts_manager.cc:194] Registered new tserver with Master: 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65:36647)
W20260808 07:00:40.539508 3131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:40.539708 3131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:40.539732 3131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:40.540899 3131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:40.540944 3131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:40.540998 3131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:40.542466 3131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:45695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=44807
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:40.542703 3131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:40.543005 3131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:40.547996 3139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.548038 3136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.548002 3137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.548374 3131 server_base.cc:1034] running on GCE node
I20260808 07:00:40.548594 3131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:40.548843 3131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:40.550007 3131 hybrid_clock.cc:648] HybridClock initialized: now 1786172440549994 us; error 47 us; skew 500 ppm
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:40.552531 3131 init.cc:377] Logged in from keytab as oryx/127.24.181.66@KRBTEST.COM (short username oryx)
I20260808 07:00:40.552953 3131 webserver.cc:460] Webserver started at http://127.24.181.66:44807/ using document root <none> and password file <none>
I20260808 07:00:40.553148 3131 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.553201 3131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.554509 3131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:40.555111 3147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.555310 3131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:40.555393 3131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "5fda071274ac444691618383b3ebeac2"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "5fd2701d1ca356bf05157f9cdc639fe9"
server_key_iv: "d00e30ba860a86d624672f0e5f496d77"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.555685 3131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.569972 3131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:40.572454 3131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:40.572589 3131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:40.572909 3131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:40.573232 3131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:40.573268 3131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.573292 3131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:40.573304 3131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.579272 3131 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:45695
I20260808 07:00:40.579334 3306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:45695 every 8 connection(s)
I20260808 07:00:40.579700 3131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
I20260808 07:00:40.588894 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 3131
I20260808 07:00:40.589061 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 2218
I20260808 07:00:40.591075 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:45127 (local address 127.24.181.126:43181)
0808 07:00:40.580915 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.580917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.581025 (+ 108us) server_negotiation.cc:407] Connection header received
0808 07:00:40.582022 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.582023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.582038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.582084 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.582857 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583420 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.584420 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.584629 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.584705 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.587474 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.587475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.587476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.587484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.589163 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.589632 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.589633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.589634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.589693 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.590004 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.590006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.590007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.590189 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.590245 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.590768 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.590883 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:00:40.591763 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:45127)
0808 07:00:40.580942 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.580954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.581922 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.582164 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.582169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.582563 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.582760 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.582764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583531 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.583532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.584330 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.584332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.584751 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.584752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.584908 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.585438 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.585448 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.587353 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.589243 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.589249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.589251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.589544 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.589771 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.589772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.589772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.589919 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.590306 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.590314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.590525 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.591192 (+ 667us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.591421 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260808 07:00:40.593327 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.24.181.66:48829)
0808 07:00:40.580629 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.580690 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.581904 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.582164 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.582169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.582564 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.582760 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.582766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583774 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.583775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.584695 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.584697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.585701 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.585702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.585768 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.586156 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.586161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.588337 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.591450 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.591451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.591451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.591643 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.592463 (+ 820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.592464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.592464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.592551 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.592917 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.592918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.592960 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.593104 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.593219 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":49,"thread_start_us":93,"threads_started":1}
I20260808 07:00:40.593411 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:47737 (local address 127.24.181.124:45385)
0808 07:00:40.580968 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.580971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.580981 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:40.582040 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.582042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.582055 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.582089 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.582880 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583674 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.584822 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.585122 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.585216 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.588513 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.588514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.588514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.588521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.591274 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.592431 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.592432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.592432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.592487 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.592761 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.592762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.592763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.592956 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.593042 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.593207 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.593294 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:00:40.593416 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:48829 (local address 127.24.181.125:32777)
0808 07:00:40.580509 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.580511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.580779 (+ 268us) server_negotiation.cc:407] Connection header received
0808 07:00:40.582040 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.582042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.582055 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.582089 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.582889 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583667 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.585266 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.585577 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.585652 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.588424 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.588425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.588425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.588432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.590478 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.591813 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.591814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.591814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.591861 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.592666 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.592667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.592668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.592826 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.592913 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.593153 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.593267 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:40.593791 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.580455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.24.181.66:47737)
0808 07:00:40.580629 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.580690 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.581904 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.582164 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.582169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.582596 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.582760 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.582766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.583774 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.583775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.584695 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.584697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.585228 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.585229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.585291 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.585670 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.585684 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.588385 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.591700 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.591701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.591701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.592349 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.592591 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.592591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.592592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.592679 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.593040 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.593040 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.593080 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.593501 (+ 421us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.593602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":38,"spinlock_wait_cycles":7040,"thread_start_us":94,"threads_started":1}
I20260808 07:00:40.594256 3312 heartbeater.cc:347] Connected to a master server at 127.24.181.125:32777
I20260808 07:00:40.594307 3307 heartbeater.cc:347] Connected to a master server at 127.24.181.124:45385
I20260808 07:00:40.594354 3312 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.594354 3307 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.594612 3307 heartbeater.cc:511] Master 127.24.181.124:45385 requested a full tablet report, sending...
I20260808 07:00:40.594612 3312 heartbeater.cc:511] Master 127.24.181.125:32777 requested a full tablet report, sending...
I20260808 07:00:40.595192 2612 ts_manager.cc:194] Registered new tserver with Master: 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:40.595206 2870 ts_manager.cc:194] Registered new tserver with Master: 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:40.602937 3310 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:40.603011 3310 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.603092 3310 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:40.603433 1734 ts_manager.cc:194] Re-registered known tserver with Master: 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:40.603988 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:45127
I20260808 07:00:40.607110 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:40399
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:43181
--webserver_port=40059
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:40.689216 3320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:40.689435 3320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:40.689471 3320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:40.690580 3320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:40.690645 3320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:40.690697 3320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:40.692160 3320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:40399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=40059
--tserver_master_addrs=127.24.181.126:43181,127.24.181.125:32777,127.24.181.124:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:40.692417 3320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:40.692636 3320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:40.697610 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.697602 3327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:40.697672 3328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:40.697749 3320 server_base.cc:1034] running on GCE node
I20260808 07:00:40.698048 3320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:40.698287 3320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:40.699427 3320 hybrid_clock.cc:648] HybridClock initialized: now 1786172440699405 us; error 39 us; skew 500 ppm
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:40.702061 3320 init.cc:377] Logged in from keytab as oryx/127.24.181.67@KRBTEST.COM (short username oryx)
I20260808 07:00:40.702484 3320 webserver.cc:460] Webserver started at http://127.24.181.67:40059/ using document root <none> and password file <none>
I20260808 07:00:40.702734 3320 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:40.702828 3320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:40.704263 3320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:40.704944 3343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.705164 3320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:40.705276 3320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
format_stamp: "Formatted at 2026-08-08 07:00:39 on dist-test-slave-nfb5"
server_key: "4c43dc9b003298d64d029c4526ac8192"
server_key_iv: "5290fdf9bc188260327914717b3c5379"
server_key_version: "encryptionkey@0"
I20260808 07:00:40.705596 3320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:40.716310 3320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:40.718561 3320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:40.718667 3320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:40.718901 3320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:40.719259 3320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:40.719295 3320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.719316 3320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:40.719363 3320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:40.725453 3320 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:40399
I20260808 07:00:40.725507 3504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:40399 every 8 connection(s)
I20260808 07:00:40.725845 3320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:40.732141 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 3320
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.124@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
Aug 08 07:00:40 dist-test-slave-nfb5 krb5kdc[1675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172440, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.125@KRBTEST.COM
I20260808 07:00:40.738529 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.727239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:36531 (local address 127.24.181.124:45385)
0808 07:00:40.727294 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.727296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.727376 (+ 80us) server_negotiation.cc:407] Connection header received
0808 07:00:40.727864 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.727865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.727873 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.727937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.728639 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729142 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730447 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730639 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730722 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.734817 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.734818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.734819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.734829 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.736743 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737426 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.737427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.737427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.737483 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737759 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.737760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.737760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.737915 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.737963 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.738343 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.738441 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:00:40.738684 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.726586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:53275)
0808 07:00:40.726736 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.726769 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.727520 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.727852 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.727853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.728333 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.728544 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.728551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729510 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.729511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730171 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.730173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730652 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730814 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.731457 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.731467 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.734127 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.737039 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.737041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.737312 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.737539 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.737540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.737675 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.738005 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.738012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.738180 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.738293 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.738456 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":73,"thread_start_us":90,"threads_started":1}
I20260808 07:00:40.738806 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.727097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.24.181.67:36531)
0808 07:00:40.727327 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.727336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.727540 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.727961 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.727962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.728324 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.728543 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.728548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729267 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.729267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730337 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.730339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730720 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730812 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.731521 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.731526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.734670 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.736867 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.736875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.736878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.737312 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.737560 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.737561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.737676 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.738037 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.738038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.738180 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.738608 (+ 428us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.738707 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":25,"thread_start_us":69,"threads_started":1}
I20260808 07:00:40.739269 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.726577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:53275 (local address 127.24.181.126:43181)
0808 07:00:40.726643 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.726645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.726867 (+ 222us) server_negotiation.cc:407] Connection header received
0808 07:00:40.727767 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.727769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.727779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.727819 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.728657 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729420 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730262 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730530 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730589 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.734237 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.734239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.734239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.734247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.736938 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737400 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.737402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.737402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.737466 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737796 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.737797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.737797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.737914 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.737977 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.739074 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.739160 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:00:40.739254 3507 heartbeater.cc:347] Connected to a master server at 127.24.181.124:45385
I20260808 07:00:40.739349 3507 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.739533 3507 heartbeater.cc:511] Master 127.24.181.124:45385 requested a full tablet report, sending...
I20260808 07:00:40.739964 2870 ts_manager.cc:194] Registered new tserver with Master: aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:40.740486 3508 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43181
I20260808 07:00:40.740550 3508 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.740617 3508 heartbeater.cc:511] Master 127.24.181.126:43181 requested a full tablet report, sending...
I20260808 07:00:40.740890 1734 ts_manager.cc:194] Re-registered known tserver with Master: aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:40.741282 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.726871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.24.181.67:59055)
0808 07:00:40.727112 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.727122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.727527 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.727763 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.727766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.728335 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:40.728545 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.728551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729491 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.729492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730336 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.730339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730802 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.730858 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.731462 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:40.731469 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:40.734633 (+ 3164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:40.737693 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.737694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.737695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.737936 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.740487 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:40.740488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:40.740488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:40.740589 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:40.740935 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:40.740936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:40.740983 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.741039 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.741150 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":61,"thread_start_us":84,"threads_started":1}
I20260808 07:00:40.741575 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.727039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:59055 (local address 127.24.181.125:32777)
0808 07:00:40.727117 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.727120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.727181 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:00:40.727627 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.727628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.727642 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.727698 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.728653 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.729409 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730419 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.730692 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.730752 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.734765 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.734767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.734767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.734776 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.737593 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.738511 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.738512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.738513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.738579 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.740692 (+ 2113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.740693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.740693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.740841 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.740905 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.741409 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.741488 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:00:40.741636 1734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:53275
I20260808 07:00:40.742139 3509 heartbeater.cc:347] Connected to a master server at 127.24.181.125:32777
I20260808 07:00:40.742183 3509 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:40.742362 3509 heartbeater.cc:511] Master 127.24.181.125:32777 requested a full tablet report, sending...
I20260808 07:00:40.742650 2612 ts_manager.cc:194] Registered new tserver with Master: aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:40.745602 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.733271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.24.181.126:43181)
0808 07:00:40.733425 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.733426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.733608 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:00:40.733633 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.733634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.733646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.733770 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.735142 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.735830 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.738019 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.738389 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.738458 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.742316 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.742329 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.742329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.742336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.744316 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.744750 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.744753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.744753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.744816 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.745046 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.745047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.745047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.745160 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.745284 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.745391 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.745473 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260808 07:00:40.747339 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.733669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.24.181.125:32777)
0808 07:00:40.734076 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.734079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.737773 (+ 3694us) server_negotiation.cc:407] Connection header received
0808 07:00:40.737795 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.737796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.737807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.737878 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.738547 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.739381 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.742071 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.742406 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.742468 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.744156 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.744158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.744158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.744166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.745838 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.746294 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.746295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.746296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.746363 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.746584 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.746585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.746586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.746741 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.746830 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.747166 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.747255 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":144,"threads_started":1}
I20260808 07:00:40.747691 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.733437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.24.181.124:45385)
0808 07:00:40.734023 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.734025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.736063 (+ 2038us) server_negotiation.cc:407] Connection header received
0808 07:00:40.736086 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.736087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.736099 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.736816 (+ 717us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.736864 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.739602 (+ 2738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.741575 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.741986 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.742757 (+ 771us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.743934 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.743935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.743936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.743942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.746093 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.746476 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.746477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.746478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.746532 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.746743 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.746744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.746744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.746869 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.747027 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.747496 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.747575 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":65,"threads_started":1}
I20260808 07:00:40.747385 1734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:40.749154 1734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:40.756831 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45695 (local address 127.0.0.1:44818)
0808 07:00:40.752741 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.752753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.752796 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.753216 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.753218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.753219 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.753259 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.753260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.754538 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.754538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.755655 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.755658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.756605 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.756606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.756665 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.756668 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.756728 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.756764 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:00:40.757196 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.24.181.66:45695)
0808 07:00:40.753049 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.753054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.753065 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:40.753108 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.753110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.753125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.753218 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.753331 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.754456 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.755749 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.756502 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.756575 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.756650 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.756756 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.757114 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260808 07:00:40.757402 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:40399 (local address 127.0.0.1:36360)
0808 07:00:40.753294 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.753305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.753346 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.753592 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.753592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.753594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.753639 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.753640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.755126 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.755127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.756140 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.756143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.757156 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.757157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.757237 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.757241 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.757296 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.757337 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":527,"threads_started":1}
I20260808 07:00:40.757682 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.24.181.67:40399)
0808 07:00:40.753449 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.753452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.753462 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:40.753495 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.753497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.753511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.753592 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.753685 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.755043 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.756238 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.757006 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.757071 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.757120 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.757568 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.757609 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":82,"threads_started":1}
I20260808 07:00:40.757917 3215 tablet_service.cc:1467] Processing CreateTablet for tablet ba950420e28342d3b1af48afb01a7941 (DEFAULT_TABLE table=test-table [id=7188a05768784a199cbdf22a8b0cb2bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:40.758056 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36647 (local address 127.0.0.1:58194)
0808 07:00:40.754589 (+ 1796us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.754599 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.754650 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.754835 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.754836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.754838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.754878 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.754880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.756033 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.756034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.756879 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.756881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.757835 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.757836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.757899 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.757903 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.757952 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.757989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1779,"thread_start_us":465,"threads_started":1}
I20260808 07:00:40.758281 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.752947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.24.181.65:36647)
0808 07:00:40.753127 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.753132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.754668 (+ 1536us) server_negotiation.cc:407] Connection header received
0808 07:00:40.754716 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.754718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.754736 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.754820 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.754986 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.755954 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.756980 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.757633 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.757686 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.757762 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.758157 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.758201 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260808 07:00:40.758338 3215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba950420e28342d3b1af48afb01a7941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:40.758340 3408 tablet_service.cc:1467] Processing CreateTablet for tablet ba950420e28342d3b1af48afb01a7941 (DEFAULT_TABLE table=test-table [id=7188a05768784a199cbdf22a8b0cb2bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:40.758715 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba950420e28342d3b1af48afb01a7941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:40.761619 3542 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Bootstrap starting.
I20260808 07:00:40.762354 3542 tablet_bootstrap.cc:654] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:40.762395 3031 tablet_service.cc:1467] Processing CreateTablet for tablet ba950420e28342d3b1af48afb01a7941 (DEFAULT_TABLE table=test-table [id=7188a05768784a199cbdf22a8b0cb2bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:40.762498 3543 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Bootstrap starting.
I20260808 07:00:40.762696 3542 log.cc:824] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:40.762799 3031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba950420e28342d3b1af48afb01a7941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:40.763236 3543 tablet_bootstrap.cc:654] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:40.763494 3543 log.cc:824] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:40.763548 3542 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: No bootstrap required, opened a new log
I20260808 07:00:40.763610 3542 ts_tablet_manager.cc:1397] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:00:40.764357 3543 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: No bootstrap required, opened a new log
I20260808 07:00:40.764402 3543 ts_tablet_manager.cc:1397] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:40.766084 3542 raft_consensus.cc:348] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.766290 3542 raft_consensus.cc:374] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:40.766338 3542 raft_consensus.cc:729] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa73f3db22cd4fe381f206cd5afffcb5, State: Initialized, Role: FOLLOWER
I20260808 07:00:40.766492 3543 raft_consensus.cc:348] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.766605 3543 raft_consensus.cc:374] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:40.766626 3543 raft_consensus.cc:729] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fda071274ac444691618383b3ebeac2, State: Initialized, Role: FOLLOWER
I20260808 07:00:40.766736 3543 consensus_queue.cc:260] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.766774 3542 consensus_queue.cc:260] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.767015 3543 ts_tablet_manager.cc:1428] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:40.767263 3542 ts_tablet_manager.cc:1428] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260808 07:00:40.767405 3508 heartbeater.cc:503] Master 127.24.181.126:43181 was elected leader, sending a full tablet report...
I20260808 07:00:40.768247 3549 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Bootstrap starting.
I20260808 07:00:40.769239 3549 tablet_bootstrap.cc:654] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:40.769572 3549 log.cc:824] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:40.770344 3549 tablet_bootstrap.cc:492] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: No bootstrap required, opened a new log
I20260808 07:00:40.770413 3549 ts_tablet_manager.cc:1397] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:40.770579 3310 heartbeater.cc:503] Master 127.24.181.126:43181 was elected leader, sending a full tablet report...
I20260808 07:00:40.771637 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.763980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47906 (local address 127.24.181.124:45385)
0808 07:00:40.764023 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.764025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.767282 (+ 3257us) server_negotiation.cc:407] Connection header received
0808 07:00:40.767400 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.767401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.767414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.767451 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.768557 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.769271 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.770545 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.771122 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.771213 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.771286 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.771512 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.771566 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:40.771878 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.763879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47906)
0808 07:00:40.767112 (+ 3233us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.767238 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.767312 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.768157 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.768158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.768160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.768461 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.768464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.769396 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.769397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.770449 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.770452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.771334 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.771335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.771407 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.771410 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.771757 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.771815 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3213}
I20260808 07:00:40.772221 2886 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260808 07:00:40.773169 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec45ae6459b4465bb4f3616db3eb32b3. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.773255 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: LEARNER
I20260808 07:00:40.773275 3549 raft_consensus.cc:348] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.773435 3549 raft_consensus.cc:374] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:40.773488 3549 raft_consensus.cc:729] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1a939cfc2541d589c3acd86e43c2dc, State: Initialized, Role: FOLLOWER
I20260808 07:00:40.773612 3549 consensus_queue.cc:260] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.773864 3549 ts_tablet_manager.cc:1428] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:40.773918 3121 heartbeater.cc:503] Master 127.24.181.126:43181 was elected leader, sending a full tablet report...
I20260808 07:00:40.807163 3558 raft_consensus.cc:482] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:40.807291 3558 raft_consensus.cc:504] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.807830 3558 leader_election.cc:288] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695), aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:40.810979 3125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.808030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:40399 (local address 127.24.181.65:57155)
0808 07:00:40.808078 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.808090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.808134 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.808418 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.808419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.808421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.808491 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.808492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.809217 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.809218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.810020 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.810023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.810679 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.810763 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.810767 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.810826 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.810871 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":40}
I20260808 07:00:40.811004 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.808234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:57155 (local address 127.24.181.67:40399)
0808 07:00:40.808296 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.808298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.808306 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:40.808331 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.808331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.808342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.808414 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.808568 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.809132 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810119 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.810591 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810665 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.810674 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.810848 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.810908 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:00:40.811235 3128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.808013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45695 (local address 127.24.181.65:47265)
0808 07:00:40.808079 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:40.808094 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:40.808154 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:40.808322 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:40.808323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:40.808325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:40.808399 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.808401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.809237 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.809238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.810025 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:40.810028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.810909 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:40.811013 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.811017 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:40.811082 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:00:40.811131 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":30}
I20260808 07:00:40.811394 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.808102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:47265 (local address 127.24.181.66:45695)
0808 07:00:40.808155 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.808158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.808167 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:40.808223 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.808224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.808234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.808304 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:40.808478 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.809151 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810124 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.810829 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.810897 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.810905 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.811253 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.811306 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:00:40.811642 3435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba950420e28342d3b1af48afb01a7941" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" is_pre_election: true
I20260808 07:00:40.811709 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba950420e28342d3b1af48afb01a7941" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fda071274ac444691618383b3ebeac2" is_pre_election: true
I20260808 07:00:40.811828 3435 raft_consensus.cc:2454] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 0.
I20260808 07:00:40.811874 3242 raft_consensus.cc:2454] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 0.
I20260808 07:00:40.812062 2961 leader_election.cc:302] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1a939cfc2541d589c3acd86e43c2dc, aa73f3db22cd4fe381f206cd5afffcb5; no voters:
I20260808 07:00:40.812202 3558 raft_consensus.cc:2781] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:40.812260 3558 raft_consensus.cc:482] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:40.812292 3558 raft_consensus.cc:3037] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:40.813015 3558 raft_consensus.cc:504] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.813174 3558 leader_election.cc:288] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 election: Requested vote from peers 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695), aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399)
I20260808 07:00:40.813339 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba950420e28342d3b1af48afb01a7941" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fda071274ac444691618383b3ebeac2"
I20260808 07:00:40.813359 3435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba950420e28342d3b1af48afb01a7941" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
I20260808 07:00:40.813422 3242 raft_consensus.cc:3037] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:40.813444 3435 raft_consensus.cc:3037] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:40.814214 3242 raft_consensus.cc:2454] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 1.
I20260808 07:00:40.814234 3435 raft_consensus.cc:2454] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 1.
I20260808 07:00:40.814440 2961 leader_election.cc:302] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1a939cfc2541d589c3acd86e43c2dc, aa73f3db22cd4fe381f206cd5afffcb5; no voters:
I20260808 07:00:40.814564 3558 raft_consensus.cc:2781] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:40.814723 3558 raft_consensus.cc:686] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 LEADER]: Becoming Leader. State: Replica: 8b1a939cfc2541d589c3acd86e43c2dc, State: Running, Role: LEADER
I20260808 07:00:40.814826 3558 consensus_queue.cc:237] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:40.815722 1734 catalog_manager.cc:5495] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65). New cstate: current_term: 1 leader_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:40.816380 2819 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: NON_VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: true } } }
I20260808 07:00:40.816490 2819 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.817430 3561 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: attempting to promote NON_VOTER 9c2c50c360664922b08f68c464f1402f to VOTER
I20260808 07:00:40.817553 3561 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } }
I20260808 07:00:40.818022 2886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 LEARNER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 07:00:40.818022 2624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Refusing update from remote peer ec45ae6459b4465bb4f3616db3eb32b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 07:00:40.818267 3561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 07:00:40.818375 3563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 07:00:40.818992 3561 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9c2c50c360664922b08f68c464f1402f (127.24.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.819177 2886 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9c2c50c360664922b08f68c464f1402f (127.24.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.819217 2624 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9c2c50c360664922b08f68c464f1402f (127.24.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820042 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820087 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820148 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.820170 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:00:40.820250 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820317 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.820472 3563 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820540 3563 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:40.820472 3564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:40.820580 3564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:00:40.830492 3314 tablet.cc:2369] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:40.833709 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.831450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.24.181.65:36647)
0808 07:00:40.831508 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.831510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.831530 (+ 20us) server_negotiation.cc:407] Connection header received
0808 07:00:40.831561 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.831561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.831578 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.831637 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:40.831749 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.832297 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.833039 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.833268 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.833333 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.833458 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:40.833550 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:40.833610 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.833639 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.833656 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:00:40.835561 3435 raft_consensus.cc:1261] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a939cfc2541d589c3acd86e43c2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:40.835561 3242 raft_consensus.cc:1261] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a939cfc2541d589c3acd86e43c2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:40.835945 3558 consensus_queue.cc:1035] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:40.836107 3559 consensus_queue.cc:1035] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:40.840323 3566 mvcc.cc:205] Tried to move back new op lower bound from 7316162317659295744 to 7316162317578268672. Current Snapshot: MvccSnapshot[applied={T|T < 7316162317659295744 or (T in {7316162317659295744})}]
I20260808 07:00:40.841152 3567 mvcc.cc:205] Tried to move back new op lower bound from 7316162317659295744 to 7316162317578268672. Current Snapshot: MvccSnapshot[applied={T|T < 7316162317659295744}]
I20260808 07:00:40.841934 1734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:40.842036 1734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:40.843312 1734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec45ae6459b4465bb4f3616db3eb32b3: Sending DeleteTablet for 3 replicas of tablet ba950420e28342d3b1af48afb01a7941
I20260808 07:00:40.843606 3215 tablet_service.cc:1514] Processing DeleteTablet for tablet ba950420e28342d3b1af48afb01a7941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:40 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 07:00:40.843647 3031 tablet_service.cc:1514] Processing DeleteTablet for tablet ba950420e28342d3b1af48afb01a7941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:40 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:58194
I20260808 07:00:40.843796 3408 tablet_service.cc:1514] Processing DeleteTablet for tablet ba950420e28342d3b1af48afb01a7941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:40 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:36360
I20260808 07:00:40.844003 3585 tablet_replica.cc:330] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: stopping tablet replica
I20260808 07:00:40.844105 3585 raft_consensus.cc:2229] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:40.844214 3585 raft_consensus.cc:2258] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:40.844456 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 1694
I20260808 07:00:40.844507 3584 tablet_replica.cc:330] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: stopping tablet replica
I20260808 07:00:40.844691 3587 tablet_replica.cc:330] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: stopping tablet replica
I20260808 07:00:40.844941 3587 raft_consensus.cc:2229] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:40.845078 3587 raft_consensus.cc:2258] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:40.848491 3587 ts_tablet_manager.cc:1905] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:40.848491 3585 ts_tablet_manager.cc:1905] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:40.844605 3584 raft_consensus.cc:2229] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:40.850229 3584 raft_consensus.cc:2258] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:40.850355 3587 ts_tablet_manager.cc:1918] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:40.850463 3587 log.cc:1196] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/ba950420e28342d3b1af48afb01a7941
I20260808 07:00:40.850777 3587 ts_tablet_manager.cc:1939] T ba950420e28342d3b1af48afb01a7941 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting consensus metadata
I20260808 07:00:40.851761 3584 ts_tablet_manager.cc:1905] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:40.853048 3584 ts_tablet_manager.cc:1918] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:40.853139 3584 log.cc:1196] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/ba950420e28342d3b1af48afb01a7941
I20260808 07:00:40.853390 3584 ts_tablet_manager.cc:1939] T ba950420e28342d3b1af48afb01a7941 P 8b1a939cfc2541d589c3acd86e43c2dc: Deleting consensus metadata
I20260808 07:00:40.853794 3585 ts_tablet_manager.cc:1918] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:40.853870 3585 log.cc:1196] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/ba950420e28342d3b1af48afb01a7941
I20260808 07:00:40.854116 3585 ts_tablet_manager.cc:1939] T ba950420e28342d3b1af48afb01a7941 P 5fda071274ac444691618383b3ebeac2: Deleting consensus metadata
I20260808 07:00:40.854558 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.844169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.24.181.124:45385)
0808 07:00:40.844256 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.844258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.844330 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:00:40.844438 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.844439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.844451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.844482 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.845654 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.846657 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.848044 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.848525 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.848579 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.850690 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.850692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.850692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.850701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.853010 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.853547 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.853549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.853549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.853621 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.853881 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.853882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.853883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.854029 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.854105 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.854325 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.854414 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 07:00:40.863335 2968 connection.cc:535] server connection from 127.0.0.1:58194 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:00:40.863467 3346 connection.cc:535] server connection from 127.0.0.1:36360 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:00:40.863750 3149 connection.cc:535] server connection from 127.0.0.1:44818 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:00:40.872526 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.864963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.24.181.124:45385)
0808 07:00:40.865010 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.865012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.865185 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:00:40.865258 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.865259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.865271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.865323 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.866185 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.866668 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.867498 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.867686 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.867772 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.869353 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.869354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.869355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.869363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.871321 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.871696 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.871697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.871698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.871756 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.871952 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.871953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.871953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.872110 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.872217 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.872343 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.872414 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:00:40.873092 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:40.865077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.24.181.125:32777)
0808 07:00:40.865125 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:40.865127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:40.865270 (+ 143us) server_negotiation.cc:407] Connection header received
0808 07:00:40.865372 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:40.865373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:40.865388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:40.865419 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:40.866224 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.867008 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.868080 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:40.868403 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:40.868501 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:40.869914 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:40.869915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:40.869915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:40.869923 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:40.871556 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.872079 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.872080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.872081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.872144 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:40.872373 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:40.872374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:40.872374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:40.872500 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:40.872576 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:40.872891 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 07:00:40.872984 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:00:41.841161 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:41.840833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:40359)
0808 07:00:41.840968 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:41.841087 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260808 07:00:41.841444 3310 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:43181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:41.842222 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:41.841946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:38973)
0808 07:00:41.842077 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:41.842157 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
W20260808 07:00:41.842458 3121 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:43181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:41.842535 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:41.842272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:49203)
0808 07:00:41.842426 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:41.842499 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
W20260808 07:00:41.842711 3508 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:43181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.343820 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec45ae6459b4465bb4f3616db3eb32b3)
I20260808 07:00:42.343969 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } }
I20260808 07:00:42.344200 3607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec45ae6459b4465bb4f3616db3eb32b3)
I20260808 07:00:42.344372 3607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } }
I20260808 07:00:42.344534 3604 leader_election.cc:288] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181), 9c2c50c360664922b08f68c464f1402f (127.24.181.124:45385)
I20260808 07:00:42.344679 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.344443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43622)
0808 07:00:42.344559 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.344632 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260808 07:00:42.344671 3607 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181), 59d128f57b0c4ad8bd635f84a9b96134 (127.24.181.125:32777)
W20260808 07:00:42.344803 2593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.345146 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.344763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43632)
0808 07:00:42.345039 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.345102 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":70,"threads_started":1}
W20260808 07:00:42.345253 2850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.345590 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.345500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43640)
0808 07:00:42.345541 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.345563 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:00:42.345676 2593 leader_election.cc:334] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181): Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.345768 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.345553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43644)
0808 07:00:42.345695 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.345729 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
W20260808 07:00:42.345849 2850 leader_election.cc:334] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181): Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.347685 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.344461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:45385 (local address 127.0.0.1:47926)
0808 07:00:42.344647 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.344658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:42.344721 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:42.344976 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:42.344977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:42.344980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:42.345047 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.345049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.346012 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.346013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.346777 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.346779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.347421 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.347422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.347490 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.347493 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:42.347569 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:00:42.347616 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":36,"threads_started":1}
I20260808 07:00:42.347692 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.344619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.24.181.124:45385)
0808 07:00:42.344828 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.344830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.344839 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:42.344867 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.344868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.344878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.345117 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:42.345158 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.345936 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.346901 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.347309 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.347420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.347429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.347576 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.347617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":47,"threads_started":1}
I20260808 07:00:42.347970 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59d128f57b0c4ad8bd635f84a9b96134" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9c2c50c360664922b08f68c464f1402f" is_pre_election: true
I20260808 07:00:42.348095 2886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59d128f57b0c4ad8bd635f84a9b96134 in term 1.
I20260808 07:00:42.348297 2586 leader_election.cc:302] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59d128f57b0c4ad8bd635f84a9b96134, 9c2c50c360664922b08f68c464f1402f; no voters: ec45ae6459b4465bb4f3616db3eb32b3
I20260808 07:00:42.348361 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.344947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:32777 (local address 127.0.0.1:47478)
0808 07:00:42.345104 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.345115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:42.345173 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:42.345528 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:42.345529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:42.345531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:42.345585 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.345587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.346549 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.346549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.347377 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.347380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.348057 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.348057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.348144 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.348148 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:42.348227 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:00:42.348272 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":41,"threads_started":1}
I20260808 07:00:42.348433 3604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:00:42.348470 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec45ae6459b4465bb4f3616db3eb32b3)
I20260808 07:00:42.348505 3604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:00:42.348515 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.345066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.24.181.125:32777)
0808 07:00:42.345245 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.345247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.345255 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:42.345405 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.345406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.345416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.345481 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:42.345787 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.346466 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.347470 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.347968 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.348050 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.348059 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.348301 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.348334 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260808 07:00:42.348683 2624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c2c50c360664922b08f68c464f1402f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59d128f57b0c4ad8bd635f84a9b96134" is_pre_election: true
I20260808 07:00:42.349367 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } }
I20260808 07:00:42.349498 2624 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2c50c360664922b08f68c464f1402f in current term 2: Already voted for candidate 59d128f57b0c4ad8bd635f84a9b96134 in this term.
I20260808 07:00:42.349519 3604 leader_election.cc:288] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 election: Requested vote from peers ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181), 9c2c50c360664922b08f68c464f1402f (127.24.181.124:45385)
I20260808 07:00:42.349745 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59d128f57b0c4ad8bd635f84a9b96134" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9c2c50c360664922b08f68c464f1402f"
I20260808 07:00:42.349766 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.349642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43654)
0808 07:00:42.349700 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.349738 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:00:42.349850 2886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:00:42.349776 2850 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2c50c360664922b08f68c464f1402f; no voters: 59d128f57b0c4ad8bd635f84a9b96134, ec45ae6459b4465bb4f3616db3eb32b3
I20260808 07:00:42.350095 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.349970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43666)
0808 07:00:42.350040 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.350066 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 07:00:42.350171 2593 leader_election.cc:334] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181): Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
I20260808 07:00:42.350571 2886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59d128f57b0c4ad8bd635f84a9b96134 in term 2.
I20260808 07:00:42.350657 3607 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:00:42.350791 2586 leader_election.cc:302] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59d128f57b0c4ad8bd635f84a9b96134, 9c2c50c360664922b08f68c464f1402f; no voters: ec45ae6459b4465bb4f3616db3eb32b3
I20260808 07:00:42.350893 3604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:00:42.351075 3604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [term 2 LEADER]: Becoming Leader. State: Replica: 59d128f57b0c4ad8bd635f84a9b96134, State: Running, Role: LEADER
I20260808 07:00:42.351159 3604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } }
I20260808 07:00:42.351507 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59d128f57b0c4ad8bd635f84a9b96134. Latest consensus state: current_term: 2 leader_uuid: "59d128f57b0c4ad8bd635f84a9b96134" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:42.351616 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:42.351836 3623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:42.352411 3623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:42.352478 3623 catalog_manager.cc:1259] Loaded cluster ID: 062349e0c0964164a4fdc95ceeca3902
I20260808 07:00:42.352533 3623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:42.352700 3623 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:42.352803 3623 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Loaded TSK: 0
I20260808 07:00:42.352949 3623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:00:42.411388 2611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:42.413149 2611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:42.414161 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.414062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43698)
0808 07:00:42.414100 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.414129 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 07:00:42.414203 2886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [term 2 FOLLOWER]: Refusing update from remote peer 59d128f57b0c4ad8bd635f84a9b96134: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 07:00:42.414551 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.414444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.0.0.1:43712)
0808 07:00:42.414479 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.414504 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 07:00:42.414489 3619 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 07:00:42.414642 2593 consensus_peers.cc:487] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 -> Peer ec45ae6459b4465bb4f3616db3eb32b3 (127.24.181.126:43181): Couldn't send request to peer ec45ae6459b4465bb4f3616db3eb32b3. Status: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:00:42.415190 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59d128f57b0c4ad8bd635f84a9b96134. Latest consensus state: current_term: 2 leader_uuid: "59d128f57b0c4ad8bd635f84a9b96134" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:42.415302 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:00:42.415750 3604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "59d128f57b0c4ad8bd635f84a9b96134" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:42.415838 3604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:42.416472 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "59d128f57b0c4ad8bd635f84a9b96134" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "ec45ae6459b4465bb4f3616db3eb32b3" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43181 } } peers { permanent_uuid: "59d128f57b0c4ad8bd635f84a9b96134" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 32777 } attrs { promote: false } } peers { permanent_uuid: "9c2c50c360664922b08f68c464f1402f" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 45385 } attrs { promote: false } } }
I20260808 07:00:42.416582 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2c50c360664922b08f68c464f1402f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:00:42.421350 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.417657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:40399 (local address 127.0.0.1:36366)
0808 07:00:42.417745 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.417755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:42.417802 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:42.418244 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:42.418245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:42.418248 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:42.418301 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.418303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.419112 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.419113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.420324 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.420326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.421103 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.421104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.421166 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.421170 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:42.421221 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:00:42.421270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":49}
I20260808 07:00:42.421452 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.417700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45695 (local address 127.0.0.1:44830)
0808 07:00:42.417754 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.417763 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:42.417802 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:42.418047 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:42.418048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:42.418050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:42.418105 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.418106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.419142 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.419142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.420313 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.420315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.421185 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.421186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.421252 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.421257 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:00:42.421311 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:00:42.421350 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":37}
I20260808 07:00:42.421484 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.417771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.24.181.66:45695)
0808 07:00:42.417892 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.417894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.417904 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:42.417935 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.417936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.417948 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.418176 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:42.418222 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.419041 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.420413 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.421103 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.421168 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.421181 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.421336 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.421389 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260808 07:00:42.421649 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.417730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.24.181.67:40399)
0808 07:00:42.418102 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.418104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.418113 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:42.418139 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.418140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.418151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.418381 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:42.418420 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.419031 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.420413 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.421015 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.421085 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.421100 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.421522 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.421583 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":72,"threads_started":1}
I20260808 07:00:42.421813 3408 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 (DEFAULT_TABLE table=test-table [id=d50b38103e0649ce961a74831605eb2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:42.421963 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a839364e347cb8502d09f6a7accc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:42.422274 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.417944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36647 (local address 127.0.0.1:58214)
0808 07:00:42.418232 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.418268 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:42.418320 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:42.418467 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:42.418468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:42.418470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:42.418508 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.418509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.420422 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.420422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.421208 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:42.421210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.422038 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.422039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:42.422109 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.422113 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:42.422175 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:00:42.422214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":84,"threads_started":1}
I20260808 07:00:42.422286 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.418004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.24.181.65:36647)
0808 07:00:42.418127 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.418130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.418311 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:00:42.418386 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.418386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.418397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.418457 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:42.418558 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.419259 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.421294 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.421936 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.422034 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.422048 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.422179 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.422213 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260808 07:00:42.422570 3031 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 (DEFAULT_TABLE table=test-table [id=d50b38103e0649ce961a74831605eb2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:42.422710 3031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a839364e347cb8502d09f6a7accc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:42.423015 3215 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 (DEFAULT_TABLE table=test-table [id=d50b38103e0649ce961a74831605eb2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:42.423166 3215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a839364e347cb8502d09f6a7accc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:42.423585 3636 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Bootstrap starting.
I20260808 07:00:42.424114 3636 tablet_bootstrap.cc:654] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:42.424160 3637 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: Bootstrap starting.
I20260808 07:00:42.424546 3637 tablet_bootstrap.cc:654] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:42.424823 3636 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: No bootstrap required, opened a new log
I20260808 07:00:42.424883 3636 ts_tablet_manager.cc:1397] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:42.425000 3641 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Bootstrap starting.
I20260808 07:00:42.425084 3637 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: No bootstrap required, opened a new log
I20260808 07:00:42.425056 3636 raft_consensus.cc:348] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.425130 3636 raft_consensus.cc:374] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:42.425145 3637 ts_tablet_manager.cc:1397] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:42.425164 3636 raft_consensus.cc:729] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa73f3db22cd4fe381f206cd5afffcb5, State: Initialized, Role: FOLLOWER
I20260808 07:00:42.425235 3636 consensus_queue.cc:260] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.425271 3637 raft_consensus.cc:348] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.425333 3637 raft_consensus.cc:374] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:42.425356 3637 raft_consensus.cc:729] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1a939cfc2541d589c3acd86e43c2dc, State: Initialized, Role: FOLLOWER
I20260808 07:00:42.425400 3637 consensus_queue.cc:260] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.425441 3636 ts_tablet_manager.cc:1428] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:42.425536 3641 tablet_bootstrap.cc:654] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:42.425549 3637 ts_tablet_manager.cc:1428] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:42.426085 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.425843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:33341)
0808 07:00:42.426010 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.426052 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260808 07:00:42.426096 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.425844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:34255)
0808 07:00:42.426009 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.426053 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260808 07:00:42.426673 3509 heartbeater.cc:503] Master 127.24.181.125:32777 was elected leader, sending a full tablet report...
I20260808 07:00:42.426731 3122 heartbeater.cc:503] Master 127.24.181.125:32777 was elected leader, sending a full tablet report...
I20260808 07:00:42.427050 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.426619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:41291)
0808 07:00:42.426994 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.427024 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359}
I20260808 07:00:42.427050 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.426678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:53211)
0808 07:00:42.427004 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.427028 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
I20260808 07:00:42.427222 3641 tablet_bootstrap.cc:492] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: No bootstrap required, opened a new log
I20260808 07:00:42.427284 3641 ts_tablet_manager.cc:1397] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:42.427448 3641 raft_consensus.cc:348] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.427501 3641 raft_consensus.cc:374] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:42.427515 3641 raft_consensus.cc:729] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fda071274ac444691618383b3ebeac2, State: Initialized, Role: FOLLOWER
I20260808 07:00:42.427577 3641 consensus_queue.cc:260] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.427770 3641 ts_tablet_manager.cc:1428] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:42.428124 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.427905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:60067)
0808 07:00:42.428044 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.428081 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260808 07:00:42.428311 3312 heartbeater.cc:503] Master 127.24.181.125:32777 was elected leader, sending a full tablet report...
I20260808 07:00:42.428464 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.428355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:43881)
0808 07:00:42.428416 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.428442 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:00:42.439464 3123 tablet.cc:2369] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:42.477618 3512 tablet.cc:2369] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:42.761035 3644 raft_consensus.cc:482] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:42.761129 3644 raft_consensus.cc:504] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.761319 3644 leader_election.cc:288] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399), 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:42.761610 3435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a839364e347cb8502d09f6a7accc3" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" is_pre_election: true
I20260808 07:00:42.761633 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a839364e347cb8502d09f6a7accc3" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fda071274ac444691618383b3ebeac2" is_pre_election: true
I20260808 07:00:42.761736 3435 raft_consensus.cc:2454] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 0.
I20260808 07:00:42.761736 3242 raft_consensus.cc:2454] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 0.
I20260808 07:00:42.761934 2964 leader_election.cc:302] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fda071274ac444691618383b3ebeac2, 8b1a939cfc2541d589c3acd86e43c2dc; no voters:
I20260808 07:00:42.762065 3644 raft_consensus.cc:2781] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:42.762120 3644 raft_consensus.cc:482] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:42.762151 3644 raft_consensus.cc:3037] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:42.762955 3644 raft_consensus.cc:504] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.763106 3644 leader_election.cc:288] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 election: Requested vote from peers aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399), 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695)
I20260808 07:00:42.763265 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a839364e347cb8502d09f6a7accc3" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fda071274ac444691618383b3ebeac2"
I20260808 07:00:42.763278 3435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a839364e347cb8502d09f6a7accc3" candidate_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa73f3db22cd4fe381f206cd5afffcb5"
I20260808 07:00:42.763372 3242 raft_consensus.cc:3037] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:42.763368 3435 raft_consensus.cc:3037] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:42.764145 3435 raft_consensus.cc:2454] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 1.
I20260808 07:00:42.764151 3242 raft_consensus.cc:2454] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a939cfc2541d589c3acd86e43c2dc in term 1.
I20260808 07:00:42.764335 2964 leader_election.cc:302] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fda071274ac444691618383b3ebeac2, 8b1a939cfc2541d589c3acd86e43c2dc; no voters:
I20260808 07:00:42.764429 3644 raft_consensus.cc:2781] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:42.764657 3644 raft_consensus.cc:686] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 LEADER]: Becoming Leader. State: Replica: 8b1a939cfc2541d589c3acd86e43c2dc, State: Running, Role: LEADER
I20260808 07:00:42.764740 3644 consensus_queue.cc:237] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } }
I20260808 07:00:42.765058 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.764929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:60069)
0808 07:00:42.764969 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.765006 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 07:00:42.765455 2611 catalog_manager.cc:5495] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65). New cstate: current_term: 1 leader_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b1a939cfc2541d589c3acd86e43c2dc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:42.773463 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.771127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.24.181.65:36647)
0808 07:00:42.771175 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:42.771177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:42.771188 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:42.771264 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:42.771265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:42.771273 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:42.771316 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:42.771446 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.771999 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.772685 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:42.772875 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:42.772926 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:42.773148 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:42.773175 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:42.773211 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:42.773361 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:00:42.773381 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:00:42.774361 3435 raft_consensus.cc:1261] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a939cfc2541d589c3acd86e43c2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:42.774384 3242 raft_consensus.cc:1261] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a939cfc2541d589c3acd86e43c2dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:42.774595 3652 consensus_queue.cc:1035] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fda071274ac444691618383b3ebeac2" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:42.774698 3644 consensus_queue.cc:1035] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa73f3db22cd4fe381f206cd5afffcb5" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:42.775377 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.775265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:51555)
0808 07:00:42.775315 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.775348 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:00:42.775920 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.775819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:54023)
0808 07:00:42.775858 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.775887 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 07:00:42.777055 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.776918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.67:43587)
0808 07:00:42.776980 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.777019 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:00:42.777484 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.776715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:47241)
0808 07:00:42.777421 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.777456 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":688}
I20260808 07:00:42.777632 2611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:42.777747 2611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:42.778609 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.778462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.66:44369)
0808 07:00:42.778553 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.778578 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 07:00:42.779201 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.779083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43181 (local address 127.24.181.65:54453)
0808 07:00:42.779124 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.779169 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:00:42.779313 2611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59d128f57b0c4ad8bd635f84a9b96134: Sending DeleteTablet for 3 replicas of tablet 2f3a839364e347cb8502d09f6a7accc3
I20260808 07:00:42.779567 3031 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:42 UTC) from {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:58214
I20260808 07:00:42.779611 3408 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:42 UTC) from {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:36366
I20260808 07:00:42.779584 3030 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:42 UTC) from {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:58214
I20260808 07:00:42.779639 3215 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a839364e347cb8502d09f6a7accc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:42 UTC) from {username='oryx', principal='oryx/127.24.181.125@KRBTEST.COM'} at 127.0.0.1:44830
I20260808 07:00:42.779774 3670 tablet_replica.cc:330] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc: stopping tablet replica
I20260808 07:00:42.779829 3670 raft_consensus.cc:2229] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:42.779893 3670 raft_consensus.cc:2258] T 2f3a839364e347cb8502d09f6a7accc3 P 8b1a939cfc2541d589c3acd86e43c2dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:42.780030 3671 tablet_replica.cc:330] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: stopping tablet replica
I20260808 07:00:42.780093 3671 raft_consensus.cc:2229] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:42.780112 3672 tablet_replica.cc:330] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: stopping tablet replica
I20260808 07:00:42.780144 3671 raft_consensus.cc:2258] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:42.780160 3672 raft_consensus.cc:2229] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:42.780201 3672 raft_consensus.cc:2258] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:42.780257 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 2814
I20260808 07:00:42.786682 3672 ts_tablet_manager.cc:1905] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:42.788535 3672 ts_tablet_manager.cc:1918] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:42.788600 3672 log.cc:1196] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/2f3a839364e347cb8502d09f6a7accc3
I20260808 07:00:42.788890 3672 ts_tablet_manager.cc:1939] T 2f3a839364e347cb8502d09f6a7accc3 P 5fda071274ac444691618383b3ebeac2: Deleting consensus metadata
I20260808 07:00:42.789017 3671 ts_tablet_manager.cc:1905] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:42.789206 2590 catalog_manager.cc:4841] TS 5fda071274ac444691618383b3ebeac2 (127.24.181.66:45695): tablet 2f3a839364e347cb8502d09f6a7accc3 (table test-table [id=d50b38103e0649ce961a74831605eb2c]) successfully deleted
I20260808 07:00:42.790364 3671 ts_tablet_manager.cc:1918] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:42.790449 3671 log.cc:1196] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/2f3a839364e347cb8502d09f6a7accc3
I20260808 07:00:42.790692 3671 ts_tablet_manager.cc:1939] T 2f3a839364e347cb8502d09f6a7accc3 P aa73f3db22cd4fe381f206cd5afffcb5: Deleting consensus metadata
I20260808 07:00:42.792026 2583 catalog_manager.cc:4841] TS aa73f3db22cd4fe381f206cd5afffcb5 (127.24.181.67:40399): tablet 2f3a839364e347cb8502d09f6a7accc3 (table test-table [id=d50b38103e0649ce961a74831605eb2c]) successfully deleted
I20260808 07:00:42.796315 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 3131
I20260808 07:00:42.803570 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:42.796093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36647 (local address 127.0.0.1:58232)
0808 07:00:42.803503 (+ 7410us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:42.803538 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7387}
W20260808 07:00:42.804504 2586 catalog_manager.cc:4584] TS 8b1a939cfc2541d589c3acd86e43c2dc (127.24.181.65:36647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f3a839364e347cb8502d09f6a7accc3: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36647: connect: Connection refused (error 111)
I20260808 07:00:42.811255 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 3320
I20260808 07:00:42.823395 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 2415
I20260808 07:00:42.833216 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 2688
2026-08-08T07:00:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5195 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:42 dist-test-slave-nfb5 krb5kdc[3678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:42 dist-test-slave-nfb5 krb5kdc[3678](info): set up 2 sockets
Aug 08 07:00:42 dist-test-slave-nfb5 krb5kdc[3678](info): commencing operation
krb5kdc: starting...
W20260808 07:00:44.084304 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab.
Entry for principal oryx/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab.
I20260808 07:00:44.206216 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41595
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab
--principal=oryx/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:44.290439 3694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:44.290652 3694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:44.290674 3694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:44.291903 3694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:44.291953 3694 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:00:44.291967 3694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:44.291980 3694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:44.291993 3694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:44.292006 3694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:44.293493 3694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41595
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.126.keytab
--principal=oryx/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:44.293694 3694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:44.293879 3694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:44.298775 3707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.298871 3705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.298813 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:44.299022 3694 server_base.cc:1034] running on GCE node
I20260808 07:00:44.299253 3694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:44.299499 3694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:44.300683 3694 hybrid_clock.cc:648] HybridClock initialized: now 1786172444300675 us; error 40 us; skew 500 ppm
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:44.303455 3694 init.cc:377] Logged in from keytab as oryx/127.24.181.126@KRBTEST.COM (short username oryx)
I20260808 07:00:44.304025 3694 webserver.cc:460] Webserver started at http://127.24.181.126:44145/ using document root <none> and password file <none>
I20260808 07:00:44.304296 3694 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:44.304373 3694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:44.304524 3694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:44.305594 3694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "46fc217d455044428e0810f99959a089"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d3c37bf6d1ab748cfc8c452da2f22311"
server_key_iv: "231a36722d2e89d8f200607eef681890"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.305979 3694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "46fc217d455044428e0810f99959a089"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d3c37bf6d1ab748cfc8c452da2f22311"
server_key_iv: "231a36722d2e89d8f200607eef681890"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.307379 3694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:44.308142 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.308317 3694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:44.308411 3694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "46fc217d455044428e0810f99959a089"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d3c37bf6d1ab748cfc8c452da2f22311"
server_key_iv: "231a36722d2e89d8f200607eef681890"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.308496 3694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:44.326294 3694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:44.328770 3694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:44.328948 3694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:44.333931 3694 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:41595
I20260808 07:00:44.333983 3804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:41595 every 8 connection(s)
I20260808 07:00:44.334350 3694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:44.335064 3805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:44.337359 3805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Bootstrap starting.
I20260808 07:00:44.337996 3805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:44.338318 3805 log.cc:824] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:44.339078 3805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: No bootstrap required, opened a new log
I20260808 07:00:44.341132 3805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } }
I20260808 07:00:44.341292 3805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:44.341315 3805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fc217d455044428e0810f99959a089, State: Initialized, Role: FOLLOWER
I20260808 07:00:44.341490 3805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } }
I20260808 07:00:44.341583 3805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:44.341622 3805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:44.341675 3805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:44.341749 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 3694
I20260808 07:00:44.341853 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:44.342067 25301 external_mini_cluster.cc:1428] Setting key f9e951dcfb815ea6d6a66f0788d8093b
I20260808 07:00:44.342422 3805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } }
I20260808 07:00:44.342569 3805 leader_election.cc:302] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46fc217d455044428e0810f99959a089; no voters:
I20260808 07:00:44.342741 3805 leader_election.cc:288] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:44.342840 3810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:44.343047 3810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [term 1 LEADER]: Becoming Leader. State: Replica: 46fc217d455044428e0810f99959a089, State: Running, Role: LEADER
I20260808 07:00:44.343160 3810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } }
I20260808 07:00:44.343047 3805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:44.343595 3812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46fc217d455044428e0810f99959a089. Latest consensus state: current_term: 1 leader_uuid: "46fc217d455044428e0810f99959a089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } } }
I20260808 07:00:44.343878 3812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:44.344125 3811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46fc217d455044428e0810f99959a089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc217d455044428e0810f99959a089" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41595 } } }
I20260808 07:00:44.344177 3816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:44.344266 3811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:44.344801 3816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:44.346336 3816 catalog_manager.cc:1347] Generated new cluster ID: 8521526b73764544bcd1dfd0188a944c
I20260808 07:00:44.346395 3816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:44.354010 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.342586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54880 (local address 127.24.181.126:41595)
0808 07:00:44.342770 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.342780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.342817 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:00:44.342990 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.342998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.343061 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.343931 (+ 870us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.344053 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.345702 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.346730 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.347351 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.347450 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.350336 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.350340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.350340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.350352 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.352356 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.352774 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.352777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.352778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.352890 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.353124 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.353125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.353126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.353412 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.353564 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.353688 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.353812 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260808 07:00:44.360632 3816 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:44.361148 3816 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:44.364674 3816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Generated new TSK 0
I20260808 07:00:44.364856 3816 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
I20260808 07:00:44.398275 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41595
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:44.505811 3835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:44.506016 3835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:44.506040 3835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:44.507218 3835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:44.507258 3835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:44.507279 3835 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:44.507297 3835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:44.508809 3835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:44.509032 3835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:44.509282 3835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:44.514581 3843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.514654 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:44.514757 3835 server_base.cc:1034] running on GCE node
W20260808 07:00:44.514575 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:44.515084 3835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:44.515334 3835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:44.516480 3835 hybrid_clock.cc:648] HybridClock initialized: now 1786172444516451 us; error 45 us; skew 500 ppm
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:44.521684 3835 init.cc:377] Logged in from keytab as oryx/127.24.181.65@KRBTEST.COM (short username oryx)
I20260808 07:00:44.522123 3835 webserver.cc:460] Webserver started at http://127.24.181.65:46357/ using document root <none> and password file <none>
I20260808 07:00:44.522348 3835 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:44.522395 3835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:44.522567 3835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:44.523650 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8cb22f76c59945168d0caea9454e5a0e"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "fd44746b46d7eb04424195987784daf5"
server_key_iv: "81d3ea9f5893c5cbf727bc0fd580cf00"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.524019 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8cb22f76c59945168d0caea9454e5a0e"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "fd44746b46d7eb04424195987784daf5"
server_key_iv: "81d3ea9f5893c5cbf727bc0fd580cf00"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.525399 3835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:00:44.526139 3856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.526352 3835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:44.526472 3835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8cb22f76c59945168d0caea9454e5a0e"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "fd44746b46d7eb04424195987784daf5"
server_key_iv: "81d3ea9f5893c5cbf727bc0fd580cf00"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.526556 3835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:44.541505 3835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:44.544057 3835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:44.544306 3835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:44.544991 3835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:44.545056 3835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.545138 3835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:44.545173 3835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:44.552456 3835 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:33901
I20260808 07:00:44.552544 4030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:33901 every 8 connection(s)
I20260808 07:00:44.552881 3835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:44.554579 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 3835
I20260808 07:00:44.554728 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:44.555020 25301 external_mini_cluster.cc:1428] Setting key d76e5e416cfdc12e686bbfb25daef0df
I20260808 07:00:44.556064 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.545394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:44339 (local address 127.24.181.126:41595)
0808 07:00:44.545448 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.545450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.545676 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:00:44.546161 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.546163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.546178 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.546276 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.547157 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.548016 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.549654 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.550002 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.550070 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.552948 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.552950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.552950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.552958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.554394 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.554843 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.554845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.554846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.554949 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.555242 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.555243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.555244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.555432 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.555499 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.555838 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.555933 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:00:44.556137 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.545390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.65:44339)
0808 07:00:44.545536 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.545575 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.546059 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.546337 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.546341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.546702 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.547039 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.547047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.548144 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.548145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.549570 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.549574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.550117 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.550118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.550314 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.550828 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.550838 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.552841 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.554493 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.554500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.554502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.554770 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.555024 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.555025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.555025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.555163 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.555512 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.555519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.555677 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.555799 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.555957 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260808 07:00:44.556689 4031 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41595
I20260808 07:00:44.556867 4031 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:44.557086 4031 heartbeater.cc:511] Master 127.24.181.126:41595 requested a full tablet report, sending...
I20260808 07:00:44.557933 3739 ts_manager.cc:194] Registered new tserver with Master: 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901)
I20260808 07:00:44.558691 3739 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:44339
I20260808 07:00:44.564095 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.557848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38269 (local address 127.24.181.126:41595)
0808 07:00:44.557920 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.557922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.558110 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:00:44.558159 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.558160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.558171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.558224 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.558629 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.559333 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.560105 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.560303 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.560393 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.561472 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.561473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.561473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.561479 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.562952 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.563343 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.563344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.563344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.563395 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.563607 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.563608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.563608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.563734 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.563787 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.563970 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.564035 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:00:44.564191 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.557893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.65:38269)
0808 07:00:44.558024 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.558036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.558105 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.558237 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.558238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.558472 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.558540 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.558543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.559445 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.559446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.559996 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.559998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.560409 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.560410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.560496 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.560943 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.560948 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.561370 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.563024 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.563025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.563025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.563266 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.563468 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.563469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.563469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.563540 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.563806 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.563807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.563864 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.563956 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.564070 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
I20260808 07:00:44.582199 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41595
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:44.665544 4050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:44.665784 4050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:44.665836 4050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:44.667054 4050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:44.667124 4050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:44.667160 4050 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:44.667201 4050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:44.668854 4050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:44.669121 4050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:44.669342 4050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:44.674615 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.674628 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.674616 4059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:44.675029 4050 server_base.cc:1034] running on GCE node
I20260808 07:00:44.675228 4050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:44.675426 4050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:44.676595 4050 hybrid_clock.cc:648] HybridClock initialized: now 1786172444676573 us; error 32 us; skew 500 ppm
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:44.679178 4050 init.cc:377] Logged in from keytab as oryx/127.24.181.66@KRBTEST.COM (short username oryx)
I20260808 07:00:44.679622 4050 webserver.cc:460] Webserver started at http://127.24.181.66:37299/ using document root <none> and password file <none>
I20260808 07:00:44.679857 4050 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:44.679929 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:44.680085 4050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:44.681149 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d57ffce8106cec2b7e3086fc691147d4"
server_key_iv: "8e648c73204d45ac220edf101a10fba5"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.681525 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d57ffce8106cec2b7e3086fc691147d4"
server_key_iv: "8e648c73204d45ac220edf101a10fba5"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.682900 4050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:00:44.683744 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.683924 4050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:44.684027 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "d57ffce8106cec2b7e3086fc691147d4"
server_key_iv: "8e648c73204d45ac220edf101a10fba5"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.684114 4050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:44.698331 4050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:44.700279 4050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:44.700445 4050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:44.700985 4050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:44.701054 4050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.701102 4050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:44.701128 4050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:44.708630 4050 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:33725
I20260808 07:00:44.708714 4223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:33725 every 8 connection(s)
I20260808 07:00:44.709048 4050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:00:44.711802 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.701503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:58419 (local address 127.24.181.126:41595)
0808 07:00:44.701553 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.701555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.701798 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:00:44.702366 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.702368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.702382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.702440 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.703272 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.703771 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.704907 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.705126 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.705206 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.708790 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.708792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.708792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.708802 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.710287 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.710705 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.710707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.710707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.710781 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.711096 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.711097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.711097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.711236 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.711318 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.711595 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.711683 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:00:44.711951 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.701494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.66:58419)
0808 07:00:44.701667 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.701702 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.702243 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.702517 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.702521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.702945 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.703154 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.703159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.703871 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.703872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.704814 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.704817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.705223 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.705224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.705390 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.705993 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.706012 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.708671 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.710363 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.710369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.710371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.710616 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.710860 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.710861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.710861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.711024 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.711309 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.711317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.711455 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.711557 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.711727 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260808 07:00:44.712510 4228 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41595
I20260808 07:00:44.712617 4228 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:44.712798 4228 heartbeater.cc:511] Master 127.24.181.126:41595 requested a full tablet report, sending...
I20260808 07:00:44.713171 3739 ts_manager.cc:194] Registered new tserver with Master: cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725)
I20260808 07:00:44.713579 3739 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:58419
I20260808 07:00:44.717506 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 4050
I20260808 07:00:44.717643 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:44.717880 25301 external_mini_cluster.cc:1428] Setting key ff55d6c23a46c601541aacd6433b6dfe
I20260808 07:00:44.719229 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.713652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:55729 (local address 127.24.181.126:41595)
0808 07:00:44.713698 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.713700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.713788 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:00:44.713839 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.713840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.713851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.713912 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.714198 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.714736 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.715472 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.715648 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.715702 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.716580 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.716580 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.716581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.716585 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.717932 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.718393 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.718393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.718394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.718468 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.718711 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.718712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.718712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.718859 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.718965 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.719110 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.719171 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:44.719379 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.713590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.66:55729)
0808 07:00:44.713729 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.713739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.713795 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.713914 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.713914 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.714069 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.714122 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.714123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.714825 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.714826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.715403 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.715404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.715725 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.715726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.715781 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.716228 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.716233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.716508 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.718032 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.718033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.718034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.718304 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.718553 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.718559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.718560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.718635 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.718952 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.718953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.719003 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.719079 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.719215 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for oryx/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
I20260808 07:00:44.745976 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41595
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:44.832698 4239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:44.832978 4239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:44.833020 4239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:44.834189 4239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:44.834257 4239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:44.834291 4239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:00:44.834333 4239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:44.835958 4239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.181.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:44.836233 4239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:44.836483 4239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:44.841588 4246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.841627 4249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:44.841660 4245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:44.841777 4239 server_base.cc:1034] running on GCE node
I20260808 07:00:44.842054 4239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:44.842301 4239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:44.843472 4239 hybrid_clock.cc:648] HybridClock initialized: now 1786172444843450 us; error 41 us; skew 500 ppm
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:44.846228 4239 init.cc:377] Logged in from keytab as oryx/127.24.181.67@KRBTEST.COM (short username oryx)
I20260808 07:00:44.846716 4239 webserver.cc:460] Webserver started at http://127.24.181.67:36521/ using document root <none> and password file <none>
I20260808 07:00:44.847024 4239 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:44.847100 4239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:44.847261 4239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:44.848325 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "dc21a504b47e4cbab34f1df7127b68a2"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "5196803a4947a5c88fd963c80ef5b9e5"
server_key_iv: "df3fdfa7e02f24af5a6c8f96c76c8980"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.848690 4239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "dc21a504b47e4cbab34f1df7127b68a2"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "5196803a4947a5c88fd963c80ef5b9e5"
server_key_iv: "df3fdfa7e02f24af5a6c8f96c76c8980"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.850162 4239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:44.851033 4259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.851223 4239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:44.851334 4239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "dc21a504b47e4cbab34f1df7127b68a2"
format_stamp: "Formatted at 2026-08-08 07:00:44 on dist-test-slave-nfb5"
server_key: "5196803a4947a5c88fd963c80ef5b9e5"
server_key_iv: "df3fdfa7e02f24af5a6c8f96c76c8980"
server_key_version: "encryptionkey@0"
I20260808 07:00:44.851426 4239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:44.875000 4239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:44.877538 4239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:44.877714 4239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:44.878396 4239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:44.878453 4239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:44.878485 4239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:44.878502 4239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:44.886062 4239 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:37417
I20260808 07:00:44.886442 4239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:44.886439 4417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:37417 every 8 connection(s)
I20260808 07:00:44.888759 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.878818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:44737 (local address 127.24.181.126:41595)
0808 07:00:44.878891 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.878894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.879225 (+ 331us) server_negotiation.cc:407] Connection header received
0808 07:00:44.880021 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.880022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.880034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.880117 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.881110 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.881609 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.882732 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.882994 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.883075 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.885828 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.885829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.885830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.885839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.887556 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.887942 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.887943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.887943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.887997 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.888261 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.888261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.888262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.888371 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.888427 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.888637 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.888695 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:44.888936 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.878853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.67:44737)
0808 07:00:44.879058 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.879108 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.879846 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.880206 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.880210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.880665 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.880989 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.880998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.881764 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.881766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.882625 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.882627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.883096 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.883097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.883226 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.883764 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.883784 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.885725 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.887633 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.887639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.887641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.887869 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.888070 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.888071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.888071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.888200 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.888428 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.888433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.888544 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.888625 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.888744 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260808 07:00:44.889431 4421 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41595
I20260808 07:00:44.889533 4421 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:44.889740 4421 heartbeater.cc:511] Master 127.24.181.126:41595 requested a full tablet report, sending...
I20260808 07:00:44.890192 3739 ts_manager.cc:194] Registered new tserver with Master: dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417)
I20260808 07:00:44.890666 3739 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:44737
I20260808 07:00:44.891981 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 4239
I20260808 07:00:44.892091 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:44.892292 25301 external_mini_cluster.cc:1428] Setting key 7bbcaa10636d8fe2a5f349e224df93cf
I20260808 07:00:44.893260 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:00:44.896584 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.890567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:55425 (local address 127.24.181.126:41595)
0808 07:00:44.890609 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.890611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.890683 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:00:44.890783 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.890784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.890794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.890863 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.891284 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.891728 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.892512 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.892785 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.892838 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.893806 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.893808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.893808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.893815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.895456 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.895923 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.895924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.895924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.895977 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.896176 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.896177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.896177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.896267 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.896327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.896476 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.896530 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:00:44.896709 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.890475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.24.181.67:55425)
0808 07:00:44.890618 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.890629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.890692 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.890868 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.890869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.891121 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:44.891205 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.891207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.891816 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.891817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.892440 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.892442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.892902 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.892903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.892984 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.893415 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:44.893420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:44.893723 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:44.895579 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.895581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.895581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.895832 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.896039 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:44.896040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:44.896040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:44.896113 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:44.896327 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:44.896328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:44.896381 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.896462 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.896574 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:00:44.917214 3809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:44.911015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.24.181.126:41595)
0808 07:00:44.911086 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.911088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.911098 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:44.911134 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.911135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.911144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.911199 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.911959 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.912515 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.913378 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.913599 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.913650 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.917064 (+ 3414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54882: BlockingRecv error: recv got EOF from 127.0.0.1:54882 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:00:44 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:44.925058 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.918033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.24.181.126:41595)
0808 07:00:44.918118 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.918120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.918229 (+ 109us) server_negotiation.cc:407] Connection header received
0808 07:00:44.918303 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.918304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.918313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.918358 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:44.918855 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.919364 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.920067 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.920268 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.920341 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.922436 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:44.922438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:44.922438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:44.922445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:44.923903 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.924277 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.924278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.924279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.924340 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:44.924542 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:44.924543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:44.924543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:44.924663 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:44.924772 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.924878 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.924957 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:00:44.926833 3739 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:44.928555 3739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:44.935463 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.931303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33901 (local address 127.0.0.1:53260)
0808 07:00:44.931435 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.931444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.931517 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.931865 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.931866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.931868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:44.931918 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.931919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.933146 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.933147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.934166 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.934169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935131 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.935216 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935245 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.935315 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.935378 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":30,"thread_start_us":95,"threads_started":1}
I20260808 07:00:44.935573 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.931154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.24.181.67:37417)
0808 07:00:44.931993 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.931997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.932006 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:44.932049 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.932050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.932061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.932148 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:44.932257 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.932932 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.934528 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935181 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935275 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935354 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.935427 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.935465 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":111,"threads_started":1}
I20260808 07:00:44.935777 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.931156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.24.181.66:33725)
0808 07:00:44.931362 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.931367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.931379 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:00:44.931422 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.931424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.931441 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.931526 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:44.931790 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.932987 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.934516 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935242 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935510 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935599 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.935627 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.935667 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
I20260808 07:00:44.935583 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.930984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:33725 (local address 127.0.0.1:34850)
0808 07:00:44.931161 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.931207 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.931273 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.931552 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.931556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.931559 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:44.931622 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.931624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.933101 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.933101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.934430 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.934431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935347 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.935407 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935411 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.935439 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.935467 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":78,"thread_start_us":63,"threads_started":1}
I20260808 07:00:44.935674 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.930997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.0.0.1:50898)
0808 07:00:44.931348 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:44.931359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:44.931416 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:44.932147 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:44.932148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:44.932150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:44.932192 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.932193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.933024 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.933025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.934419 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:44.934422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935275 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:44.935347 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935351 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:44.935412 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:00:44.935461 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":77,"thread_start_us":88,"threads_started":1}
I20260808 07:00:44.936317 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:44.931142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.24.181.65:33901)
0808 07:00:44.931692 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:44.931697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:44.931714 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:00:44.931754 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:44.931757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:44.931773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:44.931854 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:44.932009 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.933061 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.934267 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:44.935023 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:44.935097 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:44.935171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:44.936155 (+ 984us) server_negotiation.cc:299] Negotiation successful
0808 07:00:44.936200 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":85,"threads_started":1}
I20260808 07:00:44.936690 4136 tablet_service.cc:1467] Processing CreateTablet for tablet 422aa93ceadd4d00825e30b22e134240 (DEFAULT_TABLE table=test-table [id=c3db3663c7c945889d40e0020488187f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:44.936774 4327 tablet_service.cc:1467] Processing CreateTablet for tablet 422aa93ceadd4d00825e30b22e134240 (DEFAULT_TABLE table=test-table [id=c3db3663c7c945889d40e0020488187f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:44.936916 3935 tablet_service.cc:1467] Processing CreateTablet for tablet 422aa93ceadd4d00825e30b22e134240 (DEFAULT_TABLE table=test-table [id=c3db3663c7c945889d40e0020488187f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:44.937115 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422aa93ceadd4d00825e30b22e134240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:44.937119 4136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422aa93ceadd4d00825e30b22e134240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:44.937228 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 422aa93ceadd4d00825e30b22e134240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:44.939985 4458 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Bootstrap starting.
I20260808 07:00:44.940693 4458 tablet_bootstrap.cc:654] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:44.941069 4458 log.cc:824] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:44.941112 4460 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Bootstrap starting.
I20260808 07:00:44.941752 4460 tablet_bootstrap.cc:654] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:44.941970 4458 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: No bootstrap required, opened a new log
I20260808 07:00:44.942046 4458 ts_tablet_manager.cc:1397] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:44.942099 4460 log.cc:824] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:44.942461 4459 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Bootstrap starting.
I20260808 07:00:44.942836 4460 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: No bootstrap required, opened a new log
I20260808 07:00:44.942926 4460 ts_tablet_manager.cc:1397] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:44.943259 4459 tablet_bootstrap.cc:654] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:44.943641 4459 log.cc:824] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:44.944344 4458 raft_consensus.cc:348] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.944497 4459 tablet_bootstrap.cc:492] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: No bootstrap required, opened a new log
I20260808 07:00:44.944558 4459 ts_tablet_manager.cc:1397] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:44.944590 4458 raft_consensus.cc:374] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:44.944756 4458 raft_consensus.cc:729] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Initialized, Role: FOLLOWER
I20260808 07:00:44.944876 4458 consensus_queue.cc:260] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.945178 4458 ts_tablet_manager.cc:1428] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:44.945217 4421 heartbeater.cc:503] Master 127.24.181.126:41595 was elected leader, sending a full tablet report...
I20260808 07:00:44.945108 4460 raft_consensus.cc:348] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.945282 4460 raft_consensus.cc:374] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:44.945333 4460 raft_consensus.cc:729] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Initialized, Role: FOLLOWER
I20260808 07:00:44.945449 4460 consensus_queue.cc:260] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.945765 4460 ts_tablet_manager.cc:1428] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:44.945801 4031 heartbeater.cc:503] Master 127.24.181.126:41595 was elected leader, sending a full tablet report...
I20260808 07:00:44.947259 4459 raft_consensus.cc:348] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.947404 4459 raft_consensus.cc:374] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:44.947451 4459 raft_consensus.cc:729] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Initialized, Role: FOLLOWER
I20260808 07:00:44.947556 4459 consensus_queue.cc:260] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:44.947813 4459 ts_tablet_manager.cc:1428] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:00:44.947896 4228 heartbeater.cc:503] Master 127.24.181.126:41595 was elected leader, sending a full tablet report...
W20260808 07:00:44.959605 4229 tablet.cc:2369] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:45.053668 4032 tablet.cc:2369] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:00:45.137247 4422 tablet.cc:2369] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:00:45.315093 4464 raft_consensus.cc:482] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:45.315272 4464 raft_consensus.cc:504] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:45.315711 4464 leader_election.cc:288] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725), 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901)
I20260808 07:00:45.318939 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.316008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:44287 (local address 127.24.181.66:33725)
0808 07:00:45.316064 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.316066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.316258 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:00:45.316326 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.316327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.316336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.316387 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:45.316609 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.317123 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318151 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.318530 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318628 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.318638 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.318805 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.318842 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:00:45.318964 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.315927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:33725 (local address 127.24.181.67:44287)
0808 07:00:45.316169 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.316183 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.316238 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.316410 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.316411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.316424 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:45.316522 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.316524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.317221 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.317222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.318051 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.318054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.318627 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.318723 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.318726 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.318797 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.318837 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":63,"thread_start_us":109,"threads_started":1}
I20260808 07:00:45.319456 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.315927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33901 (local address 127.24.181.67:35523)
0808 07:00:45.316004 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.316018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.316106 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.316300 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.316301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.316303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:45.316414 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.316416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.317096 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.317097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.318031 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.318034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.318648 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.319266 (+ 617us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.319270 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.319338 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.319382 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:00:45.319408 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422aa93ceadd4d00825e30b22e134240" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" is_pre_election: true
I20260808 07:00:45.319468 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.315988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:35523 (local address 127.24.181.65:33901)
0808 07:00:45.316043 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.316045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.316107 (+ 62us) server_negotiation.cc:407] Connection header received
0808 07:00:45.316187 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.316189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.316196 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.316276 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:45.316510 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.317012 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318135 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.318569 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.318652 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.318659 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.319355 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.319405 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:00:45.319579 4156 raft_consensus.cc:2454] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:45.319831 4261 leader_election.cc:302] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.319790 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422aa93ceadd4d00825e30b22e134240" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e" is_pre_election: true
I20260808 07:00:45.319957 3965 raft_consensus.cc:2454] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:45.319988 4464 raft_consensus.cc:2781] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:45.320078 4464 raft_consensus.cc:482] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:45.320122 4464 raft_consensus.cc:3037] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.320935 4464 raft_consensus.cc:504] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:45.321055 4464 leader_election.cc:288] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Requested vote from peers cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725), 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901)
I20260808 07:00:45.321241 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422aa93ceadd4d00825e30b22e134240" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e"
I20260808 07:00:45.321331 3965 raft_consensus.cc:3037] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.322023 3965 raft_consensus.cc:2454] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:45.322166 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422aa93ceadd4d00825e30b22e134240" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
I20260808 07:00:45.322201 4263 leader_election.cc:302] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb22f76c59945168d0caea9454e5a0e, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.322258 4156 raft_consensus.cc:3037] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.322296 4464 raft_consensus.cc:2781] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:45.322451 4464 raft_consensus.cc:686] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 LEADER]: Becoming Leader. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Running, Role: LEADER
I20260808 07:00:45.322557 4464 consensus_queue.cc:237] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } }
I20260808 07:00:45.323180 4156 raft_consensus.cc:2454] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:45.323523 3740 catalog_manager.cc:5495] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } health_report { overall_health: UNKNOWN } } }
I20260808 07:00:45.379652 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.376688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.24.181.67:37417)
0808 07:00:45.376745 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.376748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.376772 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:00:45.376825 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.376826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.376847 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.376942 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:45.377127 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.377712 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.378601 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.378870 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.379081 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.379308 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.379414 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.379511 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.379537 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.379559 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:00:45.381659 3965 raft_consensus.cc:1261] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.381659 4156 raft_consensus.cc:1261] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.382081 4468 consensus_queue.cc:1035] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.382221 4464 consensus_queue.cc:1035] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.384447 4472 mvcc.cc:205] Tried to move back new op lower bound from 7316162336279932928 to 7316162336041627648. Current Snapshot: MvccSnapshot[applied={T|T < 7316162336279932928}]
I20260808 07:00:45.386901 3739 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:45.387040 3739 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:45.387815 3739 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Sending DeleteTablet for 3 replicas of tablet 422aa93ceadd4d00825e30b22e134240
I20260808 07:00:45.388155 4136 tablet_service.cc:1514] Processing DeleteTablet for tablet 422aa93ceadd4d00825e30b22e134240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:34850
I20260808 07:00:45.388228 3935 tablet_service.cc:1514] Processing DeleteTablet for tablet 422aa93ceadd4d00825e30b22e134240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:53260
I20260808 07:00:45.388355 4327 tablet_service.cc:1514] Processing DeleteTablet for tablet 422aa93ceadd4d00825e30b22e134240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:50898
I20260808 07:00:45.388417 4487 tablet_replica.cc:330] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: stopping tablet replica
I20260808 07:00:45.388509 4487 raft_consensus.cc:2229] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:45.388504 3739 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:00:45.388600 4487 raft_consensus.cc:2258] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 07:00:45.388629 3739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:45.388643 4488 tablet_replica.cc:330] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: stopping tablet replica
I20260808 07:00:45.388733 4488 raft_consensus.cc:2229] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:45.388815 4488 raft_consensus.cc:2258] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:45.389122 4487 ts_tablet_manager.cc:1905] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.389269 4488 ts_tablet_manager.cc:1905] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.390448 4327 tablet_service.cc:1467] Processing CreateTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 (DEFAULT_TABLE table=test-table [id=a176a7af77a9403699dc90133e54b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.390561 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a1f946f9b14b2bb5847a9d09107752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.391278 3935 tablet_service.cc:1467] Processing CreateTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 (DEFAULT_TABLE table=test-table [id=a176a7af77a9403699dc90133e54b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.391296 4487 ts_tablet_manager.cc:1918] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:45.391412 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a1f946f9b14b2bb5847a9d09107752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.391402 4136 tablet_service.cc:1467] Processing CreateTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 (DEFAULT_TABLE table=test-table [id=a176a7af77a9403699dc90133e54b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.391425 4487 log.cc:1196] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/422aa93ceadd4d00825e30b22e134240
I20260808 07:00:45.391505 4136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a1f946f9b14b2bb5847a9d09107752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.391737 4487 ts_tablet_manager.cc:1939] T 422aa93ceadd4d00825e30b22e134240 P 8cb22f76c59945168d0caea9454e5a0e: Deleting consensus metadata
I20260808 07:00:45.392217 3722 catalog_manager.cc:4841] TS 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901): tablet 422aa93ceadd4d00825e30b22e134240 (table test-table [id=c3db3663c7c945889d40e0020488187f]) successfully deleted
I20260808 07:00:45.392469 4486 tablet_replica.cc:330] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: stopping tablet replica
I20260808 07:00:45.392640 4460 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Bootstrap starting.
I20260808 07:00:45.392716 4488 ts_tablet_manager.cc:1918] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:45.392786 4488 log.cc:1196] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/422aa93ceadd4d00825e30b22e134240
I20260808 07:00:45.392946 4459 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Bootstrap starting.
I20260808 07:00:45.393049 4488 ts_tablet_manager.cc:1939] T 422aa93ceadd4d00825e30b22e134240 P dc21a504b47e4cbab34f1df7127b68a2: Deleting consensus metadata
I20260808 07:00:45.393106 4460 tablet_bootstrap.cc:654] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.393420 4459 tablet_bootstrap.cc:654] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.394165 4459 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: No bootstrap required, opened a new log
I20260808 07:00:45.394222 4459 ts_tablet_manager.cc:1397] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:45.394375 4459 raft_consensus.cc:348] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.394443 4459 raft_consensus.cc:374] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.394464 4459 raft_consensus.cc:729] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.394447 3723 catalog_manager.cc:4841] TS dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417): tablet 422aa93ceadd4d00825e30b22e134240 (table test-table [id=c3db3663c7c945889d40e0020488187f]) successfully deleted
I20260808 07:00:45.394488 4458 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Bootstrap starting.
I20260808 07:00:45.394510 4459 consensus_queue.cc:260] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.394605 4459 ts_tablet_manager.cc:1428] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:45.394987 4458 tablet_bootstrap.cc:654] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.395735 4460 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: No bootstrap required, opened a new log
I20260808 07:00:45.395803 4460 ts_tablet_manager.cc:1397] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:00:45.395970 4460 raft_consensus.cc:348] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.396056 4460 raft_consensus.cc:374] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.396088 4460 raft_consensus.cc:729] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.396144 4460 consensus_queue.cc:260] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.396271 4460 ts_tablet_manager.cc:1428] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:45.396507 4458 tablet_bootstrap.cc:492] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: No bootstrap required, opened a new log
I20260808 07:00:45.396567 4458 ts_tablet_manager.cc:1397] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:45.396731 4458 raft_consensus.cc:348] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.396831 4458 raft_consensus.cc:374] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.396857 4458 raft_consensus.cc:729] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.396899 4458 consensus_queue.cc:260] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.397022 4458 ts_tablet_manager.cc:1428] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:45.397145 4486 raft_consensus.cc:2229] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:45.397325 4486 raft_consensus.cc:2258] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:45.398043 4486 ts_tablet_manager.cc:1905] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.399739 4486 ts_tablet_manager.cc:1918] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:45.399837 4486 log.cc:1196] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/422aa93ceadd4d00825e30b22e134240
I20260808 07:00:45.400374 4486 ts_tablet_manager.cc:1939] T 422aa93ceadd4d00825e30b22e134240 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting consensus metadata
I20260808 07:00:45.400859 3721 catalog_manager.cc:4841] TS cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725): tablet 422aa93ceadd4d00825e30b22e134240 (table test-table [id=c3db3663c7c945889d40e0020488187f]) successfully deleted
I20260808 07:00:45.500397 4466 raft_consensus.cc:482] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:45.500491 4466 raft_consensus.cc:504] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.500890 4466 leader_election.cc:288] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417)
I20260808 07:00:45.504420 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.501192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:47949 (local address 127.24.181.67:37417)
0808 07:00:45.501238 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.501241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.501390 (+ 149us) server_negotiation.cc:407] Connection header received
0808 07:00:45.501448 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.501449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.501459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.501509 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:45.501730 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.502354 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.503693 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.504043 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.504133 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.504142 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.504310 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.504355 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:00:45.504441 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.501087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.24.181.66:47949)
0808 07:00:45.501315 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.501327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.501393 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.501524 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.501525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.501528 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:45.501642 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.501644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.502447 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.502448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.503561 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.503563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.504133 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.504134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.504222 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.504231 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.504302 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.504346 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260808 07:00:45.504698 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.501310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:59063 (local address 127.24.181.65:33901)
0808 07:00:45.501354 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.501356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.501364 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:45.501394 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.501395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.501405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.501457 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:45.501654 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.502157 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.503664 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.504226 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.504316 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.504326 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.504561 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.504622 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:00:45.504712 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a1f946f9b14b2bb5847a9d09107752" candidate_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc21a504b47e4cbab34f1df7127b68a2" is_pre_election: true
I20260808 07:00:45.504848 4356 raft_consensus.cc:2454] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdfbd5f20a624f12ad75494c94ae7ded in term 0.
I20260808 07:00:45.504927 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.501084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33901 (local address 127.24.181.66:59063)
0808 07:00:45.501220 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.501233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.501295 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.501481 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.501482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.501484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:45.501563 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.501566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.502247 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.502248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.503561 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.503563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.504380 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.504382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.504456 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.504460 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.504799 (+ 339us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.504842 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260808 07:00:45.505038 4076 leader_election.cc:302] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.505192 4466 raft_consensus.cc:2781] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:45.505255 4466 raft_consensus.cc:482] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:45.505208 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a1f946f9b14b2bb5847a9d09107752" candidate_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e" is_pre_election: true
I20260808 07:00:45.505278 4466 raft_consensus.cc:3037] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.505290 3965 raft_consensus.cc:2454] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdfbd5f20a624f12ad75494c94ae7ded in term 0.
I20260808 07:00:45.506016 4466 raft_consensus.cc:504] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.506168 4466 leader_election.cc:288] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [CANDIDATE]: Term 1 election: Requested vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417)
I20260808 07:00:45.506320 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a1f946f9b14b2bb5847a9d09107752" candidate_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc21a504b47e4cbab34f1df7127b68a2"
I20260808 07:00:45.506341 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a1f946f9b14b2bb5847a9d09107752" candidate_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e"
I20260808 07:00:45.506402 3965 raft_consensus.cc:3037] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.506419 4356 raft_consensus.cc:3037] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.507125 4356 raft_consensus.cc:2454] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdfbd5f20a624f12ad75494c94ae7ded in term 1.
I20260808 07:00:45.507359 4076 leader_election.cc:302] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.507454 4466 raft_consensus.cc:2781] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:45.507503 3965 raft_consensus.cc:2454] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdfbd5f20a624f12ad75494c94ae7ded in term 1.
I20260808 07:00:45.507616 4466 raft_consensus.cc:686] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 LEADER]: Becoming Leader. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Running, Role: LEADER
I20260808 07:00:45.507691 4466 consensus_queue.cc:237] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.508423 3739 catalog_manager.cc:5495] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded reported cstate change: term changed from 0 to 1, leader changed from <none> to cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66). New cstate: current_term: 1 leader_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:00:45 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172444, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.126@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:45.559098 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.551669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.24.181.126:41595)
0808 07:00:45.551849 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.551851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.551862 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:45.551882 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.551883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.551893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.551957 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:45.552467 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.553181 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.553943 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.554275 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.554349 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.556496 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:45.556498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:45.556498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:45.556505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:45.557871 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:45.558269 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:45.558271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:45.558272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:45.558335 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:45.558528 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:45.558529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:45.558529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:45.558636 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:45.558713 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.558869 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.558990 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260808 07:00:45.559194 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.551583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41595 (local address 127.0.0.1:54896)
0808 07:00:45.551734 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.551747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.551829 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.551974 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.551974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.552284 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:45.552364 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.552367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.553262 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.553262 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.553863 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.553865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.554380 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.554381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.554437 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.555030 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:45.555035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:45.556407 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:45.557980 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:45.557987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:45.557989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:45.558192 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:45.558402 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:45.558403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:45.558403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:45.558463 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:45.558703 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:45.558710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:45.558755 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.558846 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.559063 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":104,"thread_start_us":78,"threads_started":1}
I20260808 07:00:45.561100 3739 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:54896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:00:45.561307 3739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:45.562757 3935 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa440a9e2d3740e28879db63c6746c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:45.562799 4327 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa440a9e2d3740e28879db63c6746c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:45.562851 4136 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa440a9e2d3740e28879db63c6746c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:00:45.562968 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.562968 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.563055 4136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7a240fe7f343c1a4c3b5a77a025e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.564234 4459 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: Bootstrap starting.
I20260808 07:00:45.564465 4460 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: Bootstrap starting.
I20260808 07:00:45.564663 4459 tablet_bootstrap.cc:654] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.564775 4460 tablet_bootstrap.cc:654] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.565137 4458 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: Bootstrap starting.
I20260808 07:00:45.565392 4459 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: No bootstrap required, opened a new log
I20260808 07:00:45.565433 4459 ts_tablet_manager.cc:1397] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:45.565471 4458 tablet_bootstrap.cc:654] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.565589 4459 raft_consensus.cc:348] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.565675 4459 raft_consensus.cc:374] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.565688 4459 raft_consensus.cc:729] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.565742 4459 consensus_queue.cc:260] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.565852 4459 ts_tablet_manager.cc:1428] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:45.565956 4466 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.566074 4466 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.566154 4458 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: No bootstrap required, opened a new log
I20260808 07:00:45.566207 4458 ts_tablet_manager.cc:1397] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:45.566324 4458 raft_consensus.cc:348] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.566419 4458 raft_consensus.cc:374] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.566442 4458 raft_consensus.cc:729] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.566484 4458 consensus_queue.cc:260] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.566550 4460 tablet_bootstrap.cc:492] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: No bootstrap required, opened a new log
I20260808 07:00:45.566604 4460 ts_tablet_manager.cc:1397] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:00:45.566617 4458 ts_tablet_manager.cc:1428] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:45.566644 4464 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.566720 4464 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.566758 4460 raft_consensus.cc:348] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.566851 4460 raft_consensus.cc:374] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.566875 4460 raft_consensus.cc:729] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.566938 4460 consensus_queue.cc:260] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.567041 4460 ts_tablet_manager.cc:1428] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:45.567075 4465 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.567189 4465 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.821442 4464 raft_consensus.cc:482] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:45.821549 4464 raft_consensus.cc:504] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.821735 4464 leader_election.cc:288] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725)
I20260808 07:00:45.822023 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7a240fe7f343c1a4c3b5a77a025e0c" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" is_pre_election: true
I20260808 07:00:45.822034 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7a240fe7f343c1a4c3b5a77a025e0c" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e" is_pre_election: true
I20260808 07:00:45.822124 4156 raft_consensus.cc:2454] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:45.822145 3965 raft_consensus.cc:2454] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:45.822306 4261 leader_election.cc:302] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.822456 4464 raft_consensus.cc:2781] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:45.822515 4464 raft_consensus.cc:482] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:45.822546 4464 raft_consensus.cc:3037] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.823344 4464 raft_consensus.cc:504] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.823493 4464 leader_election.cc:288] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Requested vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725)
I20260808 07:00:45.823654 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7a240fe7f343c1a4c3b5a77a025e0c" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
I20260808 07:00:45.823669 3965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7a240fe7f343c1a4c3b5a77a025e0c" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e"
I20260808 07:00:45.823740 3965 raft_consensus.cc:3037] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.823746 4156 raft_consensus.cc:3037] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:45.824350 3965 raft_consensus.cc:2454] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:45.824463 4156 raft_consensus.cc:2454] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:45.824553 4263 leader_election.cc:302] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb22f76c59945168d0caea9454e5a0e, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:45.824642 4464 raft_consensus.cc:2781] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:45.824712 4464 raft_consensus.cc:686] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [term 1 LEADER]: Becoming Leader. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Running, Role: LEADER
I20260808 07:00:45.824767 4464 consensus_queue.cc:237] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.825078 4477 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: TxnStatusTablet state changed. Reason: New leader dc21a504b47e4cbab34f1df7127b68a2. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.825183 4477 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:45.825358 4517 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:00:45.825516 4517 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:00:45.825435 3739 catalog_manager.cc:5495] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:45.853569 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.850787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.0.0.1:50918)
0808 07:00:45.850867 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.850880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.850972 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.851118 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.851119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.851120 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:45.851168 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.851171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.852075 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.852077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.852645 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.852647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.853122 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.853123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.853172 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.853188 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.853403 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.853413 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.853464 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.853506 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:00:45.853585 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.850876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.24.181.67:37417)
0808 07:00:45.850953 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.850956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.850967 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:00:45.851036 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.851037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.851050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.851110 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:45.851235 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.851987 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.852734 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.853040 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.853114 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.853277 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.853312 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.853373 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.853482 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.853505 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:00:45.854766 4156 raft_consensus.cc:1261] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.854879 3965 raft_consensus.cc:1261] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.855087 4464 consensus_queue.cc:1035] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.855198 4477 consensus_queue.cc:1035] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.855770 4466 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: TxnStatusTablet state changed. Reason: New leader dc21a504b47e4cbab34f1df7127b68a2. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.855870 4466 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.855859 4465 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: TxnStatusTablet state changed. Reason: New leader dc21a504b47e4cbab34f1df7127b68a2. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.855947 4465 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.856709 4477 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.856797 4477 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:45.857340 4465 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.857426 4465 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P 8cb22f76c59945168d0caea9454e5a0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.857614 4464 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.857697 4464 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P dc21a504b47e4cbab34f1df7127b68a2: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:00:45.858144 4466 tablet_replica.cc:439] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } } }
I20260808 07:00:45.858237 4466 tablet_replica.cc:442] T 0d7a240fe7f343c1a4c3b5a77a025e0c P cdfbd5f20a624f12ad75494c94ae7ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:00:45.861698 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.858784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.24.181.66:33725)
0808 07:00:45.859036 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.859038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.859047 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:45.859089 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.859090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.859112 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.859192 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:45.859320 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.859875 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.860964 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.861144 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.861192 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.861367 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.861470 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.861527 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.861592 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.861608 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260808 07:00:45.867107 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.864502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:58223 (local address 127.24.181.67:37417)
0808 07:00:45.864570 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.864572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.864582 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:45.864612 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.864613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.864622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.864686 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:45.864821 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.865530 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.866292 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.866596 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.866669 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.866831 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.866863 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.866949 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.867019 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.867039 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:45.867317 4527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.864341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.24.181.66:58223)
0808 07:00:45.864508 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.864519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.864577 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.864694 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.864695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.864696 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:45.864743 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.864744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.865602 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.865603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.866208 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.866209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.866676 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.866677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.866724 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.866738 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.866964 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.866966 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.867016 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.867048 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260808 07:00:45.869642 3965 raft_consensus.cc:1261] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Refusing update from remote peer cdfbd5f20a624f12ad75494c94ae7ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.869760 4356 raft_consensus.cc:1261] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Refusing update from remote peer cdfbd5f20a624f12ad75494c94ae7ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:45.870059 4496 consensus_queue.cc:1035] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.870185 4466 consensus_queue.cc:1035] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:45.878434 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.875991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:38395 (local address 127.24.181.66:33725)
0808 07:00:45.876028 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.876030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.876188 (+ 158us) server_negotiation.cc:407] Connection header received
0808 07:00:45.876247 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.876248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.876260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.876323 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:45.876451 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.876902 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.877656 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.877899 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.877966 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.878162 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.878195 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.878249 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.878352 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.878374 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:00:45.878443 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.875950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:33725 (local address 127.24.181.67:38395)
0808 07:00:45.876125 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:45.876136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:45.876197 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:45.876331 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:45.876332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:45.876334 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:45.876379 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.876381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.876989 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.876990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.877567 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:45.877569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.878003 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.878004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:45.878061 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.878075 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:45.878251 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:45.878253 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:00:45.878303 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:00:45.878360 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260808 07:00:45.888481 3740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:45.888581 3740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:45.889355 3740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Sending DeleteTablet for 3 replicas of tablet 33a1f946f9b14b2bb5847a9d09107752
I20260808 07:00:45.889711 3935 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:53260
I20260808 07:00:45.889810 4327 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:50898
I20260808 07:00:45.889815 4487 tablet_replica.cc:330] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: stopping tablet replica
I20260808 07:00:45.889881 4487 raft_consensus.cc:2229] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:45.889889 4488 tablet_replica.cc:330] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: stopping tablet replica
I20260808 07:00:45.889917 4487 raft_consensus.cc:2258] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:45.889937 4488 raft_consensus.cc:2229] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:45.889969 4488 raft_consensus.cc:2258] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:45.890239 4487 ts_tablet_manager.cc:1905] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.890224 4136 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a1f946f9b14b2bb5847a9d09107752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:45 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:34850
I20260808 07:00:45.890334 4486 tablet_replica.cc:330] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: stopping tablet replica
I20260808 07:00:45.890388 4486 raft_consensus.cc:2229] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:45.890476 4486 raft_consensus.cc:2258] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:45.890551 4488 ts_tablet_manager.cc:1905] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.890765 4486 ts_tablet_manager.cc:1905] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:45.892201 4487 ts_tablet_manager.cc:1918] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:45.892210 4488 ts_tablet_manager.cc:1918] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:45.892257 4487 log.cc:1196] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/33a1f946f9b14b2bb5847a9d09107752
I20260808 07:00:45.892275 4488 log.cc:1196] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/33a1f946f9b14b2bb5847a9d09107752
I20260808 07:00:45.892519 4488 ts_tablet_manager.cc:1939] T 33a1f946f9b14b2bb5847a9d09107752 P dc21a504b47e4cbab34f1df7127b68a2: Deleting consensus metadata
I20260808 07:00:45.892520 4487 ts_tablet_manager.cc:1939] T 33a1f946f9b14b2bb5847a9d09107752 P 8cb22f76c59945168d0caea9454e5a0e: Deleting consensus metadata
I20260808 07:00:45.892710 4486 ts_tablet_manager.cc:1918] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:45.892767 4486 log.cc:1196] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/33a1f946f9b14b2bb5847a9d09107752
I20260808 07:00:45.892961 4486 ts_tablet_manager.cc:1939] T 33a1f946f9b14b2bb5847a9d09107752 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting consensus metadata
I20260808 07:00:45.893018 3722 catalog_manager.cc:4841] TS 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901): tablet 33a1f946f9b14b2bb5847a9d09107752 (table test-table [id=a176a7af77a9403699dc90133e54b79c]) successfully deleted
I20260808 07:00:45.893200 3721 catalog_manager.cc:4841] TS cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725): tablet 33a1f946f9b14b2bb5847a9d09107752 (table test-table [id=a176a7af77a9403699dc90133e54b79c]) successfully deleted
I20260808 07:00:45.897106 3723 catalog_manager.cc:4841] TS dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417): tablet 33a1f946f9b14b2bb5847a9d09107752 (table test-table [id=a176a7af77a9403699dc90133e54b79c]) successfully deleted
Aug 08 07:00:45 dist-test-slave-nfb5 krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:00:45 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:45 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:00:45.925240 4507 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:45.917925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.24.181.126:41595)
0808 07:00:45.917981 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.917983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.918157 (+ 174us) server_negotiation.cc:407] Connection header received
0808 07:00:45.918219 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.918220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.918249 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.918310 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:45.919104 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.919935 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.921058 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.921391 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.921467 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.925169 (+ 3702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54902: BlockingRecv error: recv got EOF from 127.0.0.1:54902 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 07:00:45 dist-test-slave-nfb5 krb5kdc[3678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM
I20260808 07:00:45.935278 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:45.926189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.24.181.126:41595)
0808 07:00:45.926228 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:45.926230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:45.926337 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:00:45.926385 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:45.926385 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:45.926405 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:45.926473 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:45.927139 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.927877 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.928906 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:45.929195 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:45.929285 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:45.932054 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:45.932055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:45.932055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:45.932063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:45.933808 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:45.934217 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:45.934218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:45.934218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:45.934359 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:45.934598 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:45.934598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:45.934599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:45.934795 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:45.934905 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:45.935094 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:00:45.935175 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:00:45.937184 3740 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:00:45.937371 3740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:45.938855 3935 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad (DEFAULT_TABLE table=test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.938894 4136 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad (DEFAULT_TABLE table=test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.939018 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdbe35f1ed449f7a30dd5218a7335ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.939059 4136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdbe35f1ed449f7a30dd5218a7335ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.940351 4460 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Bootstrap starting.
I20260808 07:00:45.940377 4459 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Bootstrap starting.
I20260808 07:00:45.940811 4459 tablet_bootstrap.cc:654] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.940846 4460 tablet_bootstrap.cc:654] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.941598 4327 tablet_service.cc:1467] Processing CreateTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad (DEFAULT_TABLE table=test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:45.941636 4460 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: No bootstrap required, opened a new log
I20260808 07:00:45.941679 4460 ts_tablet_manager.cc:1397] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:45.941723 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fdbe35f1ed449f7a30dd5218a7335ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:45.941826 4460 raft_consensus.cc:348] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.941902 4460 raft_consensus.cc:374] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.941921 4460 raft_consensus.cc:729] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.941965 4460 consensus_queue.cc:260] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.942049 4460 ts_tablet_manager.cc:1428] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:00:45.943629 4458 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Bootstrap starting.
I20260808 07:00:45.944075 4458 tablet_bootstrap.cc:654] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:45.944770 4458 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: No bootstrap required, opened a new log
I20260808 07:00:45.944820 4458 ts_tablet_manager.cc:1397] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:45.944972 4458 raft_consensus.cc:348] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.945034 4458 raft_consensus.cc:374] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.945052 4458 raft_consensus.cc:729] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.945091 4458 consensus_queue.cc:260] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.945168 4458 ts_tablet_manager.cc:1428] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:45.946009 4459 tablet_bootstrap.cc:492] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: No bootstrap required, opened a new log
I20260808 07:00:45.946060 4459 ts_tablet_manager.cc:1397] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 07:00:45.946199 4459 raft_consensus.cc:348] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.946260 4459 raft_consensus.cc:374] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:45.946293 4459 raft_consensus.cc:729] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Initialized, Role: FOLLOWER
I20260808 07:00:45.946341 4459 consensus_queue.cc:260] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:45.946421 4459 ts_tablet_manager.cc:1428] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:46.016847 4477 raft_consensus.cc:482] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:46.016950 4477 raft_consensus.cc:504] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.017167 4477 leader_election.cc:288] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725)
I20260808 07:00:46.017407 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdbe35f1ed449f7a30dd5218a7335ad" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" is_pre_election: true
I20260808 07:00:46.017511 4156 raft_consensus.cc:2454] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:46.017457 3964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdbe35f1ed449f7a30dd5218a7335ad" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e" is_pre_election: true
I20260808 07:00:46.017567 3964 raft_consensus.cc:2454] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 0.
I20260808 07:00:46.017732 4261 leader_election.cc:302] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:46.017830 4477 raft_consensus.cc:2781] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:46.017894 4477 raft_consensus.cc:482] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:46.017926 4477 raft_consensus.cc:3037] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.018663 4477 raft_consensus.cc:504] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.018807 4477 leader_election.cc:288] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Requested vote from peers 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901), cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725)
I20260808 07:00:46.019006 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdbe35f1ed449f7a30dd5218a7335ad" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
I20260808 07:00:46.019029 3964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fdbe35f1ed449f7a30dd5218a7335ad" candidate_uuid: "dc21a504b47e4cbab34f1df7127b68a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb22f76c59945168d0caea9454e5a0e"
I20260808 07:00:46.019091 4156 raft_consensus.cc:3037] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.019097 3964 raft_consensus.cc:3037] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.019877 4156 raft_consensus.cc:2454] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:46.019953 3964 raft_consensus.cc:2454] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc21a504b47e4cbab34f1df7127b68a2 in term 1.
I20260808 07:00:46.020073 4261 leader_election.cc:302] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdfbd5f20a624f12ad75494c94ae7ded, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:46.020187 4477 raft_consensus.cc:2781] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:46.020262 4477 raft_consensus.cc:686] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 1 LEADER]: Becoming Leader. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Running, Role: LEADER
I20260808 07:00:46.020345 4477 consensus_queue.cc:237] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.021047 3740 catalog_manager.cc:5495] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "dc21a504b47e4cbab34f1df7127b68a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } health_report { overall_health: HEALTHY } } }
I20260808 07:00:46.041262 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.039050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50928 (local address 127.24.181.67:37417)
0808 07:00:46.039115 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.039116 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.039146 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:00:46.039237 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.039238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.039246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.039286 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:46.039418 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.039880 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.040660 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.040871 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.040933 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.041071 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.041097 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.041136 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.041193 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.041209 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:00:46.042045 4156 raft_consensus.cc:1261] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:46.042089 3964 raft_consensus.cc:1261] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Refusing update from remote peer dc21a504b47e4cbab34f1df7127b68a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:46.042330 4468 consensus_queue.cc:1035] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:46.042423 4477 consensus_queue.cc:1035] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:46.044582 4473 mvcc.cc:205] Tried to move back new op lower bound from 7316162338986528768 to 7316162338899951616. Current Snapshot: MvccSnapshot[applied={T|T < 7316162338986528768}]
I20260808 07:00:46.045008 3740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:46.045104 3740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:46.045727 3740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Sending DeleteTablet for 3 replicas of tablet 9fdbe35f1ed449f7a30dd5218a7335ad
I20260808 07:00:46.046003 4136 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:34850
I20260808 07:00:46.046005 3935 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:53260
I20260808 07:00:46.046023 4327 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fdbe35f1ed449f7a30dd5218a7335ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:50898
I20260808 07:00:46.046101 4487 tablet_replica.cc:330] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: stopping tablet replica
I20260808 07:00:46.046101 4486 tablet_replica.cc:330] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: stopping tablet replica
I20260808 07:00:46.046149 4486 raft_consensus.cc:2229] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:46.046167 4487 raft_consensus.cc:2229] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:46.046192 4486 raft_consensus.cc:2258] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.046202 4487 raft_consensus.cc:2258] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.046285 4488 tablet_replica.cc:330] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: stopping tablet replica
I20260808 07:00:46.046249 3740 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:00:46.046355 4488 raft_consensus.cc:2229] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 1 LEADER]: Raft consensus shutting down.
W20260808 07:00:46.046386 3740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:00:46.046416 4488 raft_consensus.cc:2258] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.046651 4486 ts_tablet_manager.cc:1905] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:46.046752 4488 ts_tablet_manager.cc:1905] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:46.047950 4487 ts_tablet_manager.cc:1905] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:46.048202 4486 ts_tablet_manager.cc:1918] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:46.048244 4488 ts_tablet_manager.cc:1918] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:46.048290 4486 log.cc:1196] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/9fdbe35f1ed449f7a30dd5218a7335ad
I20260808 07:00:46.048313 4488 log.cc:1196] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/9fdbe35f1ed449f7a30dd5218a7335ad
I20260808 07:00:46.048529 4486 ts_tablet_manager.cc:1939] T 9fdbe35f1ed449f7a30dd5218a7335ad P cdfbd5f20a624f12ad75494c94ae7ded: Deleting consensus metadata
I20260808 07:00:46.048538 4488 ts_tablet_manager.cc:1939] T 9fdbe35f1ed449f7a30dd5218a7335ad P dc21a504b47e4cbab34f1df7127b68a2: Deleting consensus metadata
I20260808 07:00:46.048875 3721 catalog_manager.cc:4841] TS cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725): tablet 9fdbe35f1ed449f7a30dd5218a7335ad (table test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]) successfully deleted
I20260808 07:00:46.048909 3723 catalog_manager.cc:4841] TS dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417): tablet 9fdbe35f1ed449f7a30dd5218a7335ad (table test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]) successfully deleted
I20260808 07:00:46.049454 4487 ts_tablet_manager.cc:1918] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:00:46.049505 4487 log.cc:1196] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/9fdbe35f1ed449f7a30dd5218a7335ad
I20260808 07:00:46.049650 4487 ts_tablet_manager.cc:1939] T 9fdbe35f1ed449f7a30dd5218a7335ad P 8cb22f76c59945168d0caea9454e5a0e: Deleting consensus metadata
I20260808 07:00:46.050048 3722 catalog_manager.cc:4841] TS 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901): tablet 9fdbe35f1ed449f7a30dd5218a7335ad (table test-table [id=27a3198d1ceb45dfbca792cdaf52d27d]) successfully deleted
I20260808 07:00:46.050810 4136 tablet_service.cc:1467] Processing CreateTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 (DEFAULT_TABLE table=test-table [id=500bf3c64a9e4dfd98ecc1b07c39d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:46.050829 3935 tablet_service.cc:1467] Processing CreateTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 (DEFAULT_TABLE table=test-table [id=500bf3c64a9e4dfd98ecc1b07c39d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:46.050833 4327 tablet_service.cc:1467] Processing CreateTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 (DEFAULT_TABLE table=test-table [id=500bf3c64a9e4dfd98ecc1b07c39d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:00:46.050944 4136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c322fcc5974432d9b47013fd0528cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:46.050944 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c322fcc5974432d9b47013fd0528cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:46.050977 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c322fcc5974432d9b47013fd0528cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:46.051859 4458 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Bootstrap starting.
I20260808 07:00:46.052119 4460 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: Bootstrap starting.
I20260808 07:00:46.052208 4458 tablet_bootstrap.cc:654] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:46.052786 4458 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: No bootstrap required, opened a new log
I20260808 07:00:46.052834 4458 ts_tablet_manager.cc:1397] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:46.052841 4460 tablet_bootstrap.cc:654] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:46.052956 4458 raft_consensus.cc:348] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.053040 4458 raft_consensus.cc:374] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:46.053063 4458 raft_consensus.cc:729] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc21a504b47e4cbab34f1df7127b68a2, State: Initialized, Role: FOLLOWER
I20260808 07:00:46.053117 4458 consensus_queue.cc:260] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.053171 4459 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Bootstrap starting.
I20260808 07:00:46.053220 4458 ts_tablet_manager.cc:1428] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:46.053452 4459 tablet_bootstrap.cc:654] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:46.053999 4459 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: No bootstrap required, opened a new log
I20260808 07:00:46.054073 4459 ts_tablet_manager.cc:1397] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:00:46.054211 4459 raft_consensus.cc:348] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.054299 4459 raft_consensus.cc:374] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:46.054327 4459 raft_consensus.cc:729] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfbd5f20a624f12ad75494c94ae7ded, State: Initialized, Role: FOLLOWER
I20260808 07:00:46.054318 4460 tablet_bootstrap.cc:492] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: No bootstrap required, opened a new log
I20260808 07:00:46.054397 4460 ts_tablet_manager.cc:1397] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:00:46.054384 4459 consensus_queue.cc:260] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.054500 4459 ts_tablet_manager.cc:1428] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:46.054556 4460 raft_consensus.cc:348] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.054631 4460 raft_consensus.cc:374] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:46.054656 4460 raft_consensus.cc:729] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Initialized, Role: FOLLOWER
I20260808 07:00:46.054698 4460 consensus_queue.cc:260] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.054785 4460 ts_tablet_manager.cc:1428] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:46.077988 4465 raft_consensus.cc:482] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:00:46.078086 4465 raft_consensus.cc:504] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.078388 4465 leader_election.cc:288] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725), dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417)
I20260808 07:00:46.081745 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.078662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:45665 (local address 127.24.181.67:37417)
0808 07:00:46.078742 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.078745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.078853 (+ 108us) server_negotiation.cc:407] Connection header received
0808 07:00:46.078957 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.078958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.078967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.079026 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:46.079229 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.079681 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.080967 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.081363 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.081423 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.081431 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.081621 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.081670 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:00:46.081761 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.078734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:40465 (local address 127.24.181.66:33725)
0808 07:00:46.078780 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.078782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.078792 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:46.078927 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.078928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.078939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.079020 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:00:46.079170 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.079674 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.080930 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.081381 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.081476 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.081485 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.081644 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.081693 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:00:46.081770 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.078558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:33725 (local address 127.24.181.65:40465)
0808 07:00:46.078728 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:46.078741 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:46.078805 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:46.079006 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:46.079007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:46.079010 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:46.079096 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.079100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.079780 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.079781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.080838 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.080841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.081486 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.081487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.081551 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.081555 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:46.081629 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:00:46.081670 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":34,"thread_start_us":62,"threads_started":1}
I20260808 07:00:46.081777 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.078573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.24.181.65:45665)
0808 07:00:46.078765 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:46.078778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:46.078871 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:46.079053 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:46.079054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:46.079057 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:00:46.079147 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.079149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.079809 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.079810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.080870 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.080873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.081455 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.081456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.081533 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.081537 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:00:46.081590 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:00:46.081654 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":46,"thread_start_us":51,"threads_started":1}
I20260808 07:00:46.082082 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c322fcc5974432d9b47013fd0528cf2" candidate_uuid: "8cb22f76c59945168d0caea9454e5a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc21a504b47e4cbab34f1df7127b68a2" is_pre_election: true
I20260808 07:00:46.082093 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c322fcc5974432d9b47013fd0528cf2" candidate_uuid: "8cb22f76c59945168d0caea9454e5a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" is_pre_election: true
I20260808 07:00:46.082190 4156 raft_consensus.cc:2454] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb22f76c59945168d0caea9454e5a0e in term 0.
I20260808 07:00:46.082190 4356 raft_consensus.cc:2454] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb22f76c59945168d0caea9454e5a0e in term 0.
I20260808 07:00:46.082378 3862 leader_election.cc:302] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb22f76c59945168d0caea9454e5a0e, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:46.082521 4465 raft_consensus.cc:2781] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:00:46.082567 4465 raft_consensus.cc:482] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:00:46.082583 4465 raft_consensus.cc:3037] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.083356 4465 raft_consensus.cc:504] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.083501 4465 leader_election.cc:288] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [CANDIDATE]: Term 1 election: Requested vote from peers cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725), dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417)
I20260808 07:00:46.083668 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c322fcc5974432d9b47013fd0528cf2" candidate_uuid: "8cb22f76c59945168d0caea9454e5a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc21a504b47e4cbab34f1df7127b68a2"
I20260808 07:00:46.083652 4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c322fcc5974432d9b47013fd0528cf2" candidate_uuid: "8cb22f76c59945168d0caea9454e5a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdfbd5f20a624f12ad75494c94ae7ded"
I20260808 07:00:46.083738 4156 raft_consensus.cc:3037] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.083738 4356 raft_consensus.cc:3037] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:46.084508 4156 raft_consensus.cc:2454] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cb22f76c59945168d0caea9454e5a0e in term 1.
I20260808 07:00:46.084522 4356 raft_consensus.cc:2454] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cb22f76c59945168d0caea9454e5a0e in term 1.
I20260808 07:00:46.084769 3862 leader_election.cc:302] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb22f76c59945168d0caea9454e5a0e, dc21a504b47e4cbab34f1df7127b68a2; no voters:
I20260808 07:00:46.084880 4465 raft_consensus.cc:2781] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:46.085022 4465 raft_consensus.cc:686] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 1 LEADER]: Becoming Leader. State: Replica: 8cb22f76c59945168d0caea9454e5a0e, State: Running, Role: LEADER
I20260808 07:00:46.085098 4465 consensus_queue.cc:237] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } }
I20260808 07:00:46.085898 3740 catalog_manager.cc:5495] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65). New cstate: current_term: 1 leader_uuid: "8cb22f76c59945168d0caea9454e5a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb22f76c59945168d0caea9454e5a0e" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 33901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 } health_report { overall_health: UNKNOWN } } }
I20260808 07:00:46.094444 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.092196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.24.181.65:33901)
0808 07:00:46.092359 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.092360 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.092368 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:46.092409 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.092410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.092427 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.092502 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:46.092650 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.093128 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.093833 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.094026 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.094075 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.094189 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.094285 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.094338 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.094370 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.094384 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260808 07:00:46.099325 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.096994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38141 (local address 127.24.181.67:37417)
0808 07:00:46.097055 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.097057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.097065 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:46.097098 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.097099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.097107 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.097169 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:46.097316 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.097757 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.098546 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.098740 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.098832 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.099063 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.099098 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.099155 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.099227 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.099248 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:00:46.099390 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.096757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:37417 (local address 127.24.181.65:38141)
0808 07:00:46.096936 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:46.096947 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:46.096997 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:46.097172 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:46.097174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:46.097176 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:46.097234 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.097235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.097875 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.097876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.098459 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.098461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.098852 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.098853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.098955 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.098971 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.099171 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.099172 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:00:46.099221 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:00:46.099261 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260808 07:00:46.101615 4156 raft_consensus.cc:1261] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Refusing update from remote peer 8cb22f76c59945168d0caea9454e5a0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:46.101651 4356 raft_consensus.cc:1261] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Refusing update from remote peer 8cb22f76c59945168d0caea9454e5a0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:00:46.101882 4465 consensus_queue.cc:1035] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc21a504b47e4cbab34f1df7127b68a2" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:46.102056 4573 consensus_queue.cc:1035] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdfbd5f20a624f12ad75494c94ae7ded" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:00:46.103834 4474 mvcc.cc:205] Tried to move back new op lower bound from 7316162339230334976 to 7316162339165294592. Current Snapshot: MvccSnapshot[applied={T|T < 7316162339230334976}]
I20260808 07:00:46.108987 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.106817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:48479 (local address 127.24.181.65:33901)
0808 07:00:46.106855 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:46.106856 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:46.106863 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:00:46.106881 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:46.106882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:46.106891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:46.107027 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:00:46.107139 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.107573 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.108227 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.108466 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.108541 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.108744 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.108778 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.108842 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:46.108904 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:00:46.108924 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:00:46.109021 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.106659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33901 (local address 127.24.181.67:48479)
0808 07:00:46.106696 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:46.106707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:46.106746 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:46.107038 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:46.107041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:46.107043 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:00:46.107082 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.107083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.107632 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.107633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.108155 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:46.108156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:46.108540 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:46.108541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:46.108605 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:46.108612 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:00:46.108843 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:00:46.108846 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:00:46.108900 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:00:46.108949 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 07:00:46.115927 3740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:46.116029 3740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:00:46.116740 3740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46fc217d455044428e0810f99959a089: Sending DeleteTablet for 3 replicas of tablet 7c322fcc5974432d9b47013fd0528cf2
I20260808 07:00:46.117012 4327 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:50898
I20260808 07:00:46.116992 3935 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:53260
I20260808 07:00:46.117062 4136 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c322fcc5974432d9b47013fd0528cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:00:46 UTC) from {username='oryx', principal='oryx/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:34850
I20260808 07:00:46.117177 4488 tablet_replica.cc:330] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: stopping tablet replica
I20260808 07:00:46.117237 4488 raft_consensus.cc:2229] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:46.117269 4488 raft_consensus.cc:2258] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.117348 4487 tablet_replica.cc:330] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e: stopping tablet replica
I20260808 07:00:46.117408 4487 raft_consensus.cc:2229] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:00:46.117420 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 3835
I20260808 07:00:46.117492 4487 raft_consensus.cc:2258] T 7c322fcc5974432d9b47013fd0528cf2 P 8cb22f76c59945168d0caea9454e5a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.117568 4486 tablet_replica.cc:330] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: stopping tablet replica
I20260808 07:00:46.117622 4486 raft_consensus.cc:2229] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:00:46.117656 4486 raft_consensus.cc:2258] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:00:46.122251 4486 ts_tablet_manager.cc:1905] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:46.124698 4488 ts_tablet_manager.cc:1905] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:00:46.126644 4488 ts_tablet_manager.cc:1918] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:46.126731 4488 log.cc:1196] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/7c322fcc5974432d9b47013fd0528cf2
I20260808 07:00:46.127038 4488 ts_tablet_manager.cc:1939] T 7c322fcc5974432d9b47013fd0528cf2 P dc21a504b47e4cbab34f1df7127b68a2: Deleting consensus metadata
I20260808 07:00:46.127418 3723 catalog_manager.cc:4841] TS dc21a504b47e4cbab34f1df7127b68a2 (127.24.181.67:37417): tablet 7c322fcc5974432d9b47013fd0528cf2 (table test-table [id=500bf3c64a9e4dfd98ecc1b07c39d2ac]) successfully deleted
I20260808 07:00:46.128088 4486 ts_tablet_manager.cc:1918] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:00:46.128166 4486 log.cc:1196] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/7c322fcc5974432d9b47013fd0528cf2
I20260808 07:00:46.128378 4486 ts_tablet_manager.cc:1939] T 7c322fcc5974432d9b47013fd0528cf2 P cdfbd5f20a624f12ad75494c94ae7ded: Deleting consensus metadata
I20260808 07:00:46.128702 3721 catalog_manager.cc:4841] TS cdfbd5f20a624f12ad75494c94ae7ded (127.24.181.66:33725): tablet 7c322fcc5974432d9b47013fd0528cf2 (table test-table [id=500bf3c64a9e4dfd98ecc1b07c39d2ac]) successfully deleted
W20260808 07:00:46.131860 3722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:33901 (error 108)
I20260808 07:00:46.132066 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 4050
I20260808 07:00:46.136943 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:46.136671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:33901 (local address 127.0.0.1:53280)
0808 07:00:46.136825 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:46.136899 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:33901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
W20260808 07:00:46.137070 3722 catalog_manager.cc:4584] TS 8cb22f76c59945168d0caea9454e5a0e (127.24.181.65:33901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c322fcc5974432d9b47013fd0528cf2: Network error: Client connection negotiation failed: client connection to 127.24.181.65:33901: connect: Connection refused (error 111)
I20260808 07:00:46.142242 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 4239
I20260808 07:00:46.157238 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 3694
2026-08-08T07:00:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3322 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:46 dist-test-slave-nfb5 krb5kdc[4594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:46 dist-test-slave-nfb5 krb5kdc[4594](info): set up 2 sockets
Aug 08 07:00:46 dist-test-slave-nfb5 krb5kdc[4594](info): commencing operation
krb5kdc: starting...
W20260808 07:00:47.363140 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:47 dist-test-slave-nfb5 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:47.489840 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34285
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:47.572805 4622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:47.573062 4622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:47.573102 4622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:47.574257 4622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:47.574332 4622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:47.574357 4622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:47.574379 4622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:47.574409 4622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:47.576000 4622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:47.576301 4622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:47.576531 4622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:47.581503 4636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:47.581503 4632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:47.581543 4629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:47.581836 4622 server_base.cc:1034] running on GCE node
I20260808 07:00:47.582264 4622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:47.582558 4622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:47.583745 4622 hybrid_clock.cc:648] HybridClock initialized: now 1786172447583732 us; error 41 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T07:00:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1426 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:00:47 dist-test-slave-nfb5 krb5kdc[4643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:00:47 dist-test-slave-nfb5 krb5kdc[4643](info): set up 2 sockets
Aug 08 07:00:47 dist-test-slave-nfb5 krb5kdc[4643](info): commencing operation
krb5kdc: starting...
W20260808 07:00:48.815194 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:00:48 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:00:48.935853 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45549
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:45549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:49.016712 4664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.016912 4664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.016934 4664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.018066 4664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:49.018110 4664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.018123 4664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.018136 4664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:49.018149 4664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:49.019616 4664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:45549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.019862 4664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.020079 4664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:49.024935 4670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.024960 4677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.025115 4664 server_base.cc:1034] running on GCE node
W20260808 07:00:49.024935 4675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.025454 4664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.025751 4664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.026955 4664 hybrid_clock.cc:648] HybridClock initialized: now 1786172449026928 us; error 43 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.029510 4664 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:49.029969 4664 webserver.cc:460] Webserver started at http://127.24.181.126:37427/ using document root <none> and password file <none>
I20260808 07:00:49.030249 4664 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.030328 4664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.030467 4664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:49.031565 4664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c70a0d6375ca41b8b2650cf1cf95bca8"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "73f402639ecfb6b3a5a8781450831daa"
server_key_iv: "7e01a4b92c82a522cbceaca35d911d02"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.031941 4664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c70a0d6375ca41b8b2650cf1cf95bca8"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "73f402639ecfb6b3a5a8781450831daa"
server_key_iv: "7e01a4b92c82a522cbceaca35d911d02"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.033272 4664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:00:49.034006 4689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.034180 4664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:49.034284 4664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c70a0d6375ca41b8b2650cf1cf95bca8"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "73f402639ecfb6b3a5a8781450831daa"
server_key_iv: "7e01a4b92c82a522cbceaca35d911d02"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.034368 4664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:49.063974 4664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:49.065939 4664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:49.066113 4664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:49.070438 4664 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:45549
I20260808 07:00:49.070477 4776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:45549 every 8 connection(s)
I20260808 07:00:49.070880 4664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:49.071784 4778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:00:49.074034 4778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Bootstrap starting.
I20260808 07:00:49.074622 4778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Neither blocks nor log segments found. Creating new log.
I20260808 07:00:49.074952 4778 log.cc:824] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:49.075640 4778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: No bootstrap required, opened a new log
I20260808 07:00:49.077219 4778 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.077379 4778 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:00:49.077404 4778 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70a0d6375ca41b8b2650cf1cf95bca8, State: Initialized, Role: FOLLOWER
I20260808 07:00:49.077560 4778 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.077634 4778 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:49.077656 4778 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:49.077721 4778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:00:49.078222 4778 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.078343 4778 leader_election.cc:302] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70a0d6375ca41b8b2650cf1cf95bca8; no voters:
I20260808 07:00:49.078537 4778 leader_election.cc:288] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:00:49.078738 4784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:00:49.078779 4778 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:49.079147 4784 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 LEADER]: Becoming Leader. State: Replica: c70a0d6375ca41b8b2650cf1cf95bca8, State: Running, Role: LEADER
I20260808 07:00:49.079257 4784 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.079723 4786 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70a0d6375ca41b8b2650cf1cf95bca8. Latest consensus state: current_term: 1 leader_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } } }
I20260808 07:00:49.079833 4786 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:49.079877 4785 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } } }
I20260808 07:00:49.079963 4785 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:49.080292 4796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:49.080853 4796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:49.080916 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 4664
I20260808 07:00:49.081028 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:00:49.081193 25301 external_mini_cluster.cc:1428] Setting key 59de2849b4e59c998f82523e7aa93780
I20260808 07:00:49.082412 4796 catalog_manager.cc:1347] Generated new cluster ID: bb77fbb81bca40a89ede079db55468d5
I20260808 07:00:49.082477 4796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:49.096688 4796 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:00:49.097386 4796 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:49.101042 4813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.081681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.24.181.126:45549)
0808 07:00:49.081867 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:49.081876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:49.089997 (+ 8121us) server_negotiation.cc:407] Connection header received
0808 07:00:49.090125 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:49.090131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:49.090188 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:49.090332 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:49.091141 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.092503 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.093615 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.094211 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.094308 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.097281 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:49.097285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:49.097286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:49.097303 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:49.099331 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.099750 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.099753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.099753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.099866 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.100098 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.100098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.100099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.100373 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:49.100560 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:49.100634 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:00:49.100809 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260808 07:00:49.108150 4796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Generated new TSK 0
I20260808 07:00:49.108362 4796 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:00:49.138315 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:49.228534 4821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.228741 4821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.228765 4821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.229869 4821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.229905 4821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.229931 4821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:49.231353 4821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.231537 4821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.231719 4821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:49.236652 4827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.236672 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.236749 4832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.236963 4821 server_base.cc:1034] running on GCE node
I20260808 07:00:49.237144 4821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.237378 4821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.238539 4821 hybrid_clock.cc:648] HybridClock initialized: now 1786172449238518 us; error 40 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.241036 4821 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:00:49.241474 4821 webserver.cc:460] Webserver started at http://127.24.181.65:46543/ using document root <none> and password file <none>
I20260808 07:00:49.241712 4821 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.241782 4821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.241931 4821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:49.243033 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "fb339a4fdaaa49749aa33e4b2bd680f6"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "336c7bdf2006b67f0b1a3459515cd51a"
server_key_iv: "ef50c144dd7bc061779d99093134f836"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.243402 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "fb339a4fdaaa49749aa33e4b2bd680f6"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "336c7bdf2006b67f0b1a3459515cd51a"
server_key_iv: "ef50c144dd7bc061779d99093134f836"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.244740 4821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:49.245482 4843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.245659 4821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:49.245766 4821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "fb339a4fdaaa49749aa33e4b2bd680f6"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "336c7bdf2006b67f0b1a3459515cd51a"
server_key_iv: "ef50c144dd7bc061779d99093134f836"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.245841 4821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:49.257004 4821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:49.258745 4821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:49.258896 4821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:49.259166 4821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:49.259478 4821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:49.259531 4821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.259572 4821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:49.259599 4821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.265719 4821 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:40451
I20260808 07:00:49.265782 5006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:40451 every 8 connection(s)
I20260808 07:00:49.266057 4821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:49.274073 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 4821
I20260808 07:00:49.274222 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:00:49.274441 25301 external_mini_cluster.cc:1428] Setting key 194651f50a2c9c5521301e737b76ff30
I20260808 07:00:49.275153 4813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.266720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:55233 (local address 127.24.181.126:45549)
0808 07:00:49.266773 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:49.266776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:49.266988 (+ 212us) server_negotiation.cc:407] Connection header received
0808 07:00:49.267530 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:49.267532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:49.267546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:49.267596 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:49.268542 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.269082 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.270066 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.270284 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.270379 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.272411 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:49.272412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:49.272413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:49.272420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:49.273646 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.274056 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.274058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.274058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.274125 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.274429 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.274430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.274430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.274595 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:49.274679 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:49.274969 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 07:00:49.275063 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:00:49.275341 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.266671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:55233)
0808 07:00:49.266841 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:49.266882 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:49.267417 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:49.267674 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:49.267678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:49.268253 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:49.268436 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.268440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.269179 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.269180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.269964 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.269967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.270392 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.270393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.270523 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.270979 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:49.270987 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:49.272303 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:49.273721 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.273727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.273729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.273964 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.274204 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.274205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.274205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.274350 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.274666 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:49.274673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:49.274807 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 07:00:49.274920 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:00:49.275099 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260808 07:00:49.275810 5009 heartbeater.cc:347] Connected to a master server at 127.24.181.126:45549
I20260808 07:00:49.275909 5009 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:49.276103 5009 heartbeater.cc:511] Master 127.24.181.126:45549 requested a full tablet report, sending...
I20260808 07:00:49.276643 4721 ts_manager.cc:194] Registered new tserver with Master: fb339a4fdaaa49749aa33e4b2bd680f6 (127.24.181.65:40451)
I20260808 07:00:49.277339 4721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:55233
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:00:49.301527 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:49.382122 5015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.382345 5015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.382385 5015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.383529 5015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.383596 5015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.383641 5015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:49.385088 5015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.385358 5015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.385584 5015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:00:49.390575 5015 server_base.cc:1034] running on GCE node
W20260808 07:00:49.390570 5025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.390540 5021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.390558 5020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.391064 5015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.391305 5015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.392475 5015 hybrid_clock.cc:648] HybridClock initialized: now 1786172449392470 us; error 37 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.398273 5015 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:00:49.398739 5015 webserver.cc:460] Webserver started at http://127.24.181.66:37281/ using document root <none> and password file <none>
I20260808 07:00:49.399035 5015 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.399106 5015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.399262 5015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:49.400302 5015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "4669522aaab24d15bffadb4fe378c8b7"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "3712458ed429093eefc22dbcd17942fd"
server_key_iv: "2e8be4bba72fd032dad6024bee49a38d"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.400676 5015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "4669522aaab24d15bffadb4fe378c8b7"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "3712458ed429093eefc22dbcd17942fd"
server_key_iv: "2e8be4bba72fd032dad6024bee49a38d"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.402029 5015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:49.402742 5033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.402962 5015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:49.403059 5015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "4669522aaab24d15bffadb4fe378c8b7"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "3712458ed429093eefc22dbcd17942fd"
server_key_iv: "2e8be4bba72fd032dad6024bee49a38d"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.403115 5015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:49.419291 5015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:49.421758 5015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:49.421880 5015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:49.422073 5015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:49.422410 5015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:49.422448 5015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.422495 5015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:49.422523 5015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.428411 5015 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:34241
I20260808 07:00:49.428499 5195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:34241 every 8 connection(s)
I20260808 07:00:49.428870 5015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:49.436340 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5015
I20260808 07:00:49.436465 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:00:49.436720 25301 external_mini_cluster.cc:1428] Setting key 1d386fa4fe032314c5e80796fb5368d7
I20260808 07:00:49.438741 4813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.429613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:38561 (local address 127.24.181.126:45549)
0808 07:00:49.429683 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:49.429686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:49.429901 (+ 215us) server_negotiation.cc:407] Connection header received
0808 07:00:49.430447 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:49.430449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:49.430463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:49.430512 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:49.431495 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.432005 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.433219 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.433409 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.433495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.435709 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:49.435710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:49.435711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:49.435719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:49.437250 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.437722 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.437723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.437724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.437786 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.438091 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.438092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.438092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.438205 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:49.438284 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:49.438568 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:00:49.438651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:00:49.438953 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.429574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:38561)
0808 07:00:49.429769 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:49.429809 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:49.430334 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:49.430581 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:49.430585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:49.431188 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:49.431391 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.431396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.432120 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.432121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.432952 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.432954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.433502 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.433503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.433637 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.434168 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:49.434177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:49.435608 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:49.437346 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.437355 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.437357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.437643 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.437876 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.437877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.437878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.438010 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.438274 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:49.438282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:49.438429 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 07:00:49.438535 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:00:49.438693 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260808 07:00:49.439431 5196 heartbeater.cc:347] Connected to a master server at 127.24.181.126:45549
I20260808 07:00:49.439538 5196 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:49.439719 5196 heartbeater.cc:511] Master 127.24.181.126:45549 requested a full tablet report, sending...
I20260808 07:00:49.440143 4721 ts_manager.cc:194] Registered new tserver with Master: 4669522aaab24d15bffadb4fe378c8b7 (127.24.181.66:34241)
I20260808 07:00:49.440523 4721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:38561
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:00:49.461613 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:49.569911 5201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.570117 5201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.570143 5201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.571462 5201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.571504 5201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.571533 5201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:49.573076 5201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.573323 5201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.573560 5201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:49.578967 5210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.579074 5201 server_base.cc:1034] running on GCE node
W20260808 07:00:49.578985 5208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.578987 5207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.579483 5201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.579756 5201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.580920 5201 hybrid_clock.cc:648] HybridClock initialized: now 1786172449580905 us; error 29 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.587713 5201 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:00:49.588240 5201 webserver.cc:460] Webserver started at http://127.24.181.67:45751/ using document root <none> and password file <none>
I20260808 07:00:49.588482 5201 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.588553 5201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.588718 5201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:00:49.589797 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "c2ea4a488bb2486aabe98d0d820f8742"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "9179bab91299f1bb696d357b2b7c7e12"
server_key_iv: "f44efa0cdc273afc259eeb8555c84420"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.590183 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "c2ea4a488bb2486aabe98d0d820f8742"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "9179bab91299f1bb696d357b2b7c7e12"
server_key_iv: "f44efa0cdc273afc259eeb8555c84420"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.591636 5201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:00:49.592387 5218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.592561 5201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:49.592664 5201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "c2ea4a488bb2486aabe98d0d820f8742"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "9179bab91299f1bb696d357b2b7c7e12"
server_key_iv: "f44efa0cdc273afc259eeb8555c84420"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.592751 5201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:49.601624 5201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:49.603509 5201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:49.603669 5201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:49.603900 5201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:49.604210 5201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:49.604266 5201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.604307 5201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:49.604336 5201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.610863 5201 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:39341
I20260808 07:00:49.610913 5394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:39341 every 8 connection(s)
I20260808 07:00:49.611270 5201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:00:49.617166 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5201
I20260808 07:00:49.617274 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:00:49.617437 25301 external_mini_cluster.cc:1428] Setting key bb53909338b3db9143471f5101565438
I20260808 07:00:49.620365 4813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.612061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:44401 (local address 127.24.181.126:45549)
0808 07:00:49.612129 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:49.612131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:49.612320 (+ 189us) server_negotiation.cc:407] Connection header received
0808 07:00:49.612769 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:49.612770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:49.612780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:49.612854 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:49.613724 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.614213 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.615239 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.615419 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.615473 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.617705 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:49.617707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:49.617707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:49.617713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:49.619211 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.619573 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.619574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.619574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.619625 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:49.619864 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:49.619865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:49.619865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:49.619962 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:49.620013 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:49.620232 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:00:49.620294 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:00:49.620502 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:49.612034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:44401)
0808 07:00:49.612186 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:49.612237 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:49.612689 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:49.612906 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:49.612909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:49.613465 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:49.613648 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.613652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.614294 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.614295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.615161 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:49.615164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:49.615493 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:49.615493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:49.615583 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:49.616058 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:49.616066 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:49.617590 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:00:49.619282 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.619288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.619290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.619505 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.619684 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:00:49.619685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:00:49.619685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:00:49.619806 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:00:49.620018 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:00:49.620022 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:00:49.620123 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 07:00:49.620203 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:00:49.620322 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260808 07:00:49.620977 5395 heartbeater.cc:347] Connected to a master server at 127.24.181.126:45549
I20260808 07:00:49.621081 5395 heartbeater.cc:464] Registering TS with master...
I20260808 07:00:49.621253 5395 heartbeater.cc:511] Master 127.24.181.126:45549 requested a full tablet report, sending...
I20260808 07:00:49.621646 4721 ts_manager.cc:194] Registered new tserver with Master: c2ea4a488bb2486aabe98d0d820f8742 (127.24.181.67:39341)
I20260808 07:00:49.622037 4721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:44401
I20260808 07:00:49.628986 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
Entry for principal oryx/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab.
WARNING: no policy specified for oryx/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
Entry for principal oryx/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab.
WARNING: no policy specified for oryx/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
Entry for principal oryx/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab.
I20260808 07:00:49.701294 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 4821
I20260808 07:00:49.723150 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5015
I20260808 07:00:49.733186 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5201
I20260808 07:00:49.751928 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 4664
I20260808 07:00:49.762567 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45549
--webserver_interface=127.24.181.126
--webserver_port=37427
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:45549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:00:49.843748 5406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.843943 5406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.843966 5406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.845108 5406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:00:49.845151 5406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.845166 5406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.845208 5406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:00:49.845240 5406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:00:49.846721 5406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:45549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.847010 5406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.847259 5406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:49.851962 5412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.851977 5414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.852090 5406 server_base.cc:1034] running on GCE node
W20260808 07:00:49.852188 5411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.852478 5406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.852739 5406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.853878 5406 hybrid_clock.cc:648] HybridClock initialized: now 1786172449853867 us; error 28 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.856362 5406 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:00:49.856783 5406 webserver.cc:460] Webserver started at http://127.24.181.126:37427/ using document root <none> and password file <none>
I20260808 07:00:49.857095 5406 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.857215 5406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.858824 5406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:49.859544 5421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.859740 5406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:49.859815 5406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c70a0d6375ca41b8b2650cf1cf95bca8"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "73f402639ecfb6b3a5a8781450831daa"
server_key_iv: "7e01a4b92c82a522cbceaca35d911d02"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.860126 5406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:49.881920 5406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:49.884007 5406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:49.884125 5406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:49.887954 5406 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:45549
I20260808 07:00:49.888015 5505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:45549 every 8 connection(s)
I20260808 07:00:49.888403 5406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:00:49.889916 5507 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:00:49.890511 5507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Bootstrap starting.
I20260808 07:00:49.897682 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5406
I20260808 07:00:49.898057 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:40451
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:45549
--webserver_port=46543
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:00:49.899509 5507 log.cc:824] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:00:49.901244 5507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:00:49.901536 5507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Bootstrap complete.
I20260808 07:00:49.903954 5507 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.904197 5507 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70a0d6375ca41b8b2650cf1cf95bca8, State: Initialized, Role: FOLLOWER
I20260808 07:00:49.904412 5507 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.904525 5507 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:00:49.904579 5507 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:00:49.904626 5507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:00:49.905556 5507 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.905675 5507 leader_election.cc:302] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70a0d6375ca41b8b2650cf1cf95bca8; no voters:
I20260808 07:00:49.905871 5507 leader_election.cc:288] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:00:49.905905 5511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:00:49.906030 5511 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [term 2 LEADER]: Becoming Leader. State: Replica: c70a0d6375ca41b8b2650cf1cf95bca8, State: Running, Role: LEADER
I20260808 07:00:49.906134 5507 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:00:49.906183 5511 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } }
I20260808 07:00:49.906523 5514 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } } }
I20260808 07:00:49.906605 5514 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:49.906940 5516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70a0d6375ca41b8b2650cf1cf95bca8. Latest consensus state: current_term: 2 leader_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70a0d6375ca41b8b2650cf1cf95bca8" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45549 } } }
I20260808 07:00:49.907035 5516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:00:49.907457 5522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:00:49.908095 5522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:00:49.908252 5522 catalog_manager.cc:1259] Loaded cluster ID: bb77fbb81bca40a89ede079db55468d5
I20260808 07:00:49.908301 5522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:00:49.909634 5522 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:00:49.909803 5522 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c70a0d6375ca41b8b2650cf1cf95bca8: Loaded TSK: 0
I20260808 07:00:49.910024 5522 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:00:49.980989 5509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:49.981195 5509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:49.981220 5509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:49.982492 5509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:49.982535 5509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:49.982563 5509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:00:49.984117 5509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.65.keytab
--principal=oryx/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:40451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=46543
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:49.984359 5509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:49.984617 5509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:49.990705 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.990875 5509 server_base.cc:1034] running on GCE node
W20260808 07:00:49.990684 5545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:49.990833 5550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:49.991246 5509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:49.991503 5509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:49.992677 5509 hybrid_clock.cc:648] HybridClock initialized: now 1786172449992666 us; error 35 us; skew 500 ppm
Aug 08 07:00:49 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172449, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:49.995435 5509 init.cc:377] Logged in from keytab as oryx/127.24.181.65@KRBTEST.COM (short username oryx)
I20260808 07:00:49.995931 5509 webserver.cc:460] Webserver started at http://127.24.181.65:46543/ using document root <none> and password file <none>
I20260808 07:00:49.996155 5509 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:49.996239 5509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:49.997627 5509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:49.998281 5561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:49.998445 5509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:49.998553 5509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "fb339a4fdaaa49749aa33e4b2bd680f6"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "336c7bdf2006b67f0b1a3459515cd51a"
server_key_iv: "ef50c144dd7bc061779d99093134f836"
server_key_version: "encryptionkey@0"
I20260808 07:00:49.998899 5509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:50.026310 5509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:50.028517 5509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:50.028669 5509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:50.028967 5509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:50.029376 5509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:50.029414 5509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.029438 5509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:50.029451 5509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.035872 5509 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:40451
I20260808 07:00:50.035931 5717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:40451 every 8 connection(s)
I20260808 07:00:50.036314 5509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:00:50.045249 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5509
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.045636 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:34241
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:45549
--webserver_port=37281
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.047173 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.037366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:53551)
0808 07:00:50.037694 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.037754 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.038500 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.039106 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.039110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.040092 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.040389 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.040397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.042256 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.042257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.043227 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.043231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.043823 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.043824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.043949 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.044559 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.044568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.046924 (+ 2356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":88,"threads_started":1}
W20260808 07:00:50.047222 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.037653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:53551 (local address 127.24.181.126:45549)
0808 07:00:50.037896 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.037906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.037953 (+ 47us) server_negotiation.cc:407] Connection header received
0808 07:00:50.038762 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.038770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.038845 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.039015 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.040515 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.042155 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.043330 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.043724 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.043850 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.047068 (+ 3218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:53551: BlockingRecv error: recv got EOF from 127.24.181.65:53551 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
W20260808 07:00:50.047546 5718 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:45549 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.181.126:45549: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.052160 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.047912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:49913)
0808 07:00:50.048023 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.048033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.048086 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.048223 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.048224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.048375 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.048428 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.048430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.049046 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.049048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.049553 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.049555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.049926 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.049927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.049996 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.050447 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.050452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.052115 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260808 07:00:50.052395 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.047976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:49913 (local address 127.24.181.126:45549)
0808 07:00:50.048014 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.048016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.048071 (+ 55us) server_negotiation.cc:407] Connection header received
0808 07:00:50.048134 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.048135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.048146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.048203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.048501 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.048963 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.049649 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.049854 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.049928 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.052245 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:49913: BlockingRecv error: recv got EOF from 127.24.181.65:49913 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.058151 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.052651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:47639)
0808 07:00:50.052712 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.052722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.052806 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.052949 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.052950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.053130 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.053159 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.053162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.054102 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.054103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.055040 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.055042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.055563 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.055564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.055652 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.056315 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.056321 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.058107 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 07:00:50.058269 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.052612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:47639 (local address 127.24.181.126:45549)
0808 07:00:50.052674 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.052676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.052777 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:00:50.052857 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.052858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.052872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.052946 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.053256 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.054021 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.055140 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.055459 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.055543 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.058209 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:47639: BlockingRecv error: recv got EOF from 127.24.181.65:47639 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 07:00:50.058472 5718 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:00:50.131086 5726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:50.131342 5726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:50.131384 5726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:50.132550 5726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:50.132619 5726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:50.132664 5726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:00:50.134100 5726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.66.keytab
--principal=oryx/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:34241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=37281
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:50.134384 5726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:50.134613 5726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:50.139612 5734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:50.139660 5726 server_base.cc:1034] running on GCE node
W20260808 07:00:50.139621 5736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:50.139654 5732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:50.140102 5726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:50.140348 5726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:50.141523 5726 hybrid_clock.cc:648] HybridClock initialized: now 1786172450141512 us; error 29 us; skew 500 ppm
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172450, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:50.144086 5726 init.cc:377] Logged in from keytab as oryx/127.24.181.66@KRBTEST.COM (short username oryx)
I20260808 07:00:50.144455 5726 webserver.cc:460] Webserver started at http://127.24.181.66:37281/ using document root <none> and password file <none>
I20260808 07:00:50.144639 5726 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:50.144690 5726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:50.145949 5726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:00:50.146497 5750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.146677 5726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:00:50.146761 5726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "4669522aaab24d15bffadb4fe378c8b7"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "3712458ed429093eefc22dbcd17942fd"
server_key_iv: "2e8be4bba72fd032dad6024bee49a38d"
server_key_version: "encryptionkey@0"
I20260808 07:00:50.147118 5726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:50.163301 5726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:50.165190 5726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:50.165311 5726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:50.165490 5726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:50.165797 5726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:50.165839 5726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.165863 5726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:50.165910 5726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.172308 5726 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:34241
I20260808 07:00:50.172392 5912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:34241 every 8 connection(s)
I20260808 07:00:50.172726 5726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:00:50.179720 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.173484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:50319 (local address 127.24.181.126:45549)
0808 07:00:50.173550 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.173552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.173732 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:00:50.174208 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.174209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.174224 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.174274 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.175218 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.175695 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.176617 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.176810 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.176860 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.179657 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:50319: BlockingRecv error: recv got EOF from 127.24.181.66:50319 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:00:50.179769 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.173465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:50319)
0808 07:00:50.173624 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.173660 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.174122 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.174326 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.174330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.174888 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.175139 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.175143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.175780 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.175781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.176542 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.176544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.176884 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.176885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.176983 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.177419 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.177427 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.179579 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
W20260808 07:00:50.179988 5913 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:45549 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.181.126:45549: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
I20260808 07:00:50.181833 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5726
I20260808 07:00:50.182156 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:39341
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:45549
--webserver_port=45751
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.184867 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.180233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:48879)
0808 07:00:50.180337 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.180347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.180395 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.180523 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.180524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.180662 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.180705 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.180707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.181314 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.181316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.181965 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.181967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.182456 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.182457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.182530 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.182928 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.182934 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.184782 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260808 07:00:50.184937 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.180295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:48879 (local address 127.24.181.126:45549)
0808 07:00:50.180335 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.180337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.180380 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:00:50.180455 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.180456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.180464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.180519 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.180787 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.181236 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.182049 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.182393 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.182452 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.184867 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:48879: BlockingRecv error: recv got EOF from 127.24.181.66:48879 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.190133 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.185288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:49405)
0808 07:00:50.185406 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.185418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.185471 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.185592 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.185593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.185747 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.185782 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.185784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.186569 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.186571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.187513 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.187516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.187846 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.187848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.187912 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.188274 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.188280 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.190053 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260808 07:00:50.190236 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.185357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:49405 (local address 127.24.181.126:45549)
0808 07:00:50.185409 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.185411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.185457 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:00:50.185525 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.185526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.185533 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.185577 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.185861 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.186482 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.187599 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.187772 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.187830 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.190126 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:49405: BlockingRecv error: recv got EOF from 127.24.181.66:49405 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 07:00:50.190398 5913 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:00:50.266532 5917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:00:50.266778 5917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:00:50.266819 5917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:00:50.268036 5917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:00:50.268103 5917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:00:50.268147 5917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:00:50.269591 5917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/krb5kdc/oryx_127.24.181.67.keytab
--principal=oryx/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:39341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=45751
--tserver_master_addrs=127.24.181.126:45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:00:50.269861 5917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:00:50.270078 5917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:00:50.275202 5923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:00:50.275271 5926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:50.275315 5917 server_base.cc:1034] running on GCE node
W20260808 07:00:50.275245 5924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:00:50.275619 5917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:00:50.275887 5917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:00:50.277087 5917 hybrid_clock.cc:648] HybridClock initialized: now 1786172450277082 us; error 38 us; skew 500 ppm
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172450, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:00:50.279652 5917 init.cc:377] Logged in from keytab as oryx/127.24.181.67@KRBTEST.COM (short username oryx)
I20260808 07:00:50.280036 5917 webserver.cc:460] Webserver started at http://127.24.181.67:45751/ using document root <none> and password file <none>
I20260808 07:00:50.280225 5917 fs_manager.cc:359] Metadata directory not provided
I20260808 07:00:50.280278 5917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:00:50.281616 5917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:50.282191 5936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.282397 5917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:00:50.282500 5917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "c2ea4a488bb2486aabe98d0d820f8742"
format_stamp: "Formatted at 2026-08-08 07:00:49 on dist-test-slave-nfb5"
server_key: "9179bab91299f1bb696d357b2b7c7e12"
server_key_iv: "f44efa0cdc273afc259eeb8555c84420"
server_key_version: "encryptionkey@0"
I20260808 07:00:50.283006 5917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:00:50.295714 5917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:00:50.298092 5917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:00:50.298261 5917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:00:50.298506 5917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:00:50.298848 5917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:00:50.298913 5917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.298977 5917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:00:50.299006 5917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:00:50.305038 5917 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:39341
I20260808 07:00:50.305119 6108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:39341 every 8 connection(s)
I20260808 07:00:50.305471 5917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:00:50.308288 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 5917
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:00:50.313268 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.306257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:48345 (local address 127.24.181.126:45549)
0808 07:00:50.306343 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.306345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.306553 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:00:50.307086 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.307087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.307097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.307190 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.308190 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.308909 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.309915 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.310129 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.310204 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.313209 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:48345: BlockingRecv error: recv got EOF from 127.24.181.67:48345 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:00:50.313357 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.306266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:48345)
0808 07:00:50.306422 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.306459 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.306983 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.307302 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.307306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.307896 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.308085 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.308089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.309027 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.309028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.309825 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.309827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.310225 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.310226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.310362 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.310890 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.310898 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.313146 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260808 07:00:50.313613 6109 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:45549 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.181.126:45549: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
I20260808 07:00:50.316064 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.308802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.24.181.126:45549)
0808 07:00:50.308912 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.308914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.308923 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:50.308997 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.308998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.309008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.309086 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.309696 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.310174 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.311183 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.311399 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.311489 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.312714 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:00:50.312723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:00:50.312723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:00:50.312733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:00:50.314607 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:50.315019 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:50.315023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:50.315023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:50.315126 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:00:50.315368 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:00:50.315369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:00:50.315369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:00:50.315599 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:00:50.315769 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 07:00:50.315840 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:00:50.315953 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.318378 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.313937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:43213)
0808 07:00:50.314024 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.314034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.314080 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.314200 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.314201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.314331 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.314371 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.314372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.315014 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.315015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.315489 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.315491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.315852 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.315853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.315918 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.316596 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.316600 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.318337 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 07:00:50.318512 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.313983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:43213 (local address 127.24.181.126:45549)
0808 07:00:50.314020 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.314022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.314065 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:00:50.314135 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.314136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.314145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.314187 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.314448 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.314891 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.315576 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.315762 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.315851 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.318443 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:43213: BlockingRecv error: recv got EOF from 127.24.181.67:43213 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:50 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:50.322187 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:50.318626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:60497)
0808 07:00:50.318664 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:50.318673 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:50.318713 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:50.318819 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:50.318819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:50.318970 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:50.319007 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.319008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.319545 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.319546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.320008 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:50.320010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.320301 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.320302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:50.320336 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.320677 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:50.320681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:50.322145 (+ 1464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:00:50.322279 5724 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:50.318681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:60497 (local address 127.24.181.126:45549)
0808 07:00:50.318728 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:50.318730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:50.318737 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:00:50.318764 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:50.318764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:50.318771 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:50.318816 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:50.319066 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.319478 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.320075 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:50.320240 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:50.320287 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:50.322225 (+ 1938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:60497: BlockingRecv error: recv got EOF from 127.24.181.67:60497 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 07:00:50.322328 6109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:51.063699 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:51.058936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:59007)
0808 07:00:51.059083 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:51.059095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:51.059160 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:51.059343 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:51.059344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:51.059513 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:51.059564 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.059565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.060210 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.060210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.060768 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.060769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.061124 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.061124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.061165 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.061619 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:51.061623 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:51.063593 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260808 07:00:51.063766 6115 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:51.059034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:59007 (local address 127.24.181.126:45549)
0808 07:00:51.059176 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:51.059178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:51.059212 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:00:51.059232 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:51.059233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:51.059243 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:51.059295 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:51.059642 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.060140 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.060858 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.061056 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.061112 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.063676 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:59007: BlockingRecv error: recv got EOF from 127.24.181.65:59007 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:51.195639 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:51.190770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:43153)
0808 07:00:51.190908 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:51.190954 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:51.191011 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:51.191139 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:51.191140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:51.191304 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:51.191362 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.191364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.192031 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.192031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.192571 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.192572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.192918 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.192919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.192959 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.193403 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:51.193408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:51.195575 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
W20260808 07:00:51.195742 6115 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:51.190883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43153 (local address 127.24.181.126:45549)
0808 07:00:51.190954 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:51.190956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:51.191008 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:00:51.191054 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:51.191055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:51.191066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:51.191122 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:51.191440 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.191959 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.192641 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.192854 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.192907 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.195675 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:43153: BlockingRecv error: recv got EOF from 127.24.181.66:43153 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:51 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:51.328639 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:51.322756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:43035)
0808 07:00:51.322981 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:51.322995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:51.323095 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:51.323294 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:51.323295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:51.323500 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:51.323551 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.323553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.324321 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.324323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.325270 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:51.325274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.325790 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.325791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:51.325856 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.326567 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:51.326573 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:51.328532 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
W20260808 07:00:51.328677 6115 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:51.322868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:43035 (local address 127.24.181.126:45549)
0808 07:00:51.322970 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:51.322972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:51.323075 (+ 103us) server_negotiation.cc:407] Connection header received
0808 07:00:51.323197 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:51.323198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:51.323218 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:51.323287 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:51.323644 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.324220 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.325374 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:51.325722 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:51.325792 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:51.328612 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:43035: BlockingRecv error: recv got EOF from 127.24.181.67:43035 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:52.069178 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:52.064170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:38305)
0808 07:00:52.064332 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:52.064343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:52.064444 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:52.064713 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:52.064715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:52.064946 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:52.065009 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.065011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.065672 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.065674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.066253 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.066255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.066632 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.066634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.066696 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.067190 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:52.067194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:52.069120 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260808 07:00:52.069339 6120 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:52.064246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38305 (local address 127.24.181.126:45549)
0808 07:00:52.064353 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:52.064356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:52.064422 (+ 66us) server_negotiation.cc:407] Connection header received
0808 07:00:52.064538 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:52.064539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:52.064553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:52.064652 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:52.065075 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.065579 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.066347 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.066553 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.066634 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.069243 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:38305: BlockingRecv error: recv got EOF from 127.24.181.65:38305 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:52.200882 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:52.196161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:40807)
0808 07:00:52.196340 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:52.196352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:52.196438 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:52.196609 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:52.196610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:52.196796 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:52.196852 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.196853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.197534 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.197536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.198076 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.198078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.198460 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.198461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.198523 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.199063 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:52.199068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:52.200824 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
W20260808 07:00:52.201025 6120 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:52.196248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:40807 (local address 127.24.181.126:45549)
0808 07:00:52.196337 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:52.196339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:52.196418 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:00:52.196511 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:52.196512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:52.196526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:52.196602 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:52.196942 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.197453 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.198180 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.198384 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.198462 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.200942 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:40807: BlockingRecv error: recv got EOF from 127.24.181.66:40807 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:52 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:52.333977 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:52.329330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:37137)
0808 07:00:52.329479 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:52.329490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:52.329539 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:52.329676 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:52.329677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:52.329866 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:52.329917 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.329919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.330534 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.330535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.331154 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:52.331157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.331453 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.331454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:52.331492 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.331957 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:52.331962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:52.333921 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":62,"threads_started":1}
W20260808 07:00:52.334106 6120 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:52.329494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:37137 (local address 127.24.181.126:45549)
0808 07:00:52.329565 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:52.329566 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:52.329574 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:52.329595 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:52.329596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:52.329607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:52.329656 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:52.329984 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.330466 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.331225 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:52.331394 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:52.331443 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:52.334021 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:37137: BlockingRecv error: recv got EOF from 127.24.181.67:37137 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:53.074716 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:53.069830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:53207)
0808 07:00:53.070018 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:53.070030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:53.070101 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:53.070312 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:53.070314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:53.070529 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:53.070582 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.070584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.071331 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.071331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.071892 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.071893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.072232 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.072232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.072276 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.072740 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:53.072745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:53.074647 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
W20260808 07:00:53.074896 6126 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:53.069932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:53207 (local address 127.24.181.126:45549)
0808 07:00:53.070053 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:53.070055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:53.070086 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:00:53.070175 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:53.070176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:53.070190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:53.070292 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:53.070658 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.071258 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.071972 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.072163 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.072219 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.074811 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:53207: BlockingRecv error: recv got EOF from 127.24.181.65:53207 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:53.206303 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:53.201414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:58743)
0808 07:00:53.201585 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:53.201596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:53.201713 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:53.201854 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:53.201855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:53.202081 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:53.202135 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.202137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.202810 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.202811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.203419 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.203421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.203809 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.203810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.203854 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.204311 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:53.204315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:53.206171 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
W20260808 07:00:53.206488 6126 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:53.201473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:58743 (local address 127.24.181.126:45549)
0808 07:00:53.201541 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:53.201544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:53.201727 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:00:53.201760 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:53.201761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:53.201774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:53.201834 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:53.202215 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.202720 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.203509 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.203737 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.203801 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.206362 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:58743: BlockingRecv error: recv got EOF from 127.24.181.66:58743 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:53 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:53.339483 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:53.334585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:37439)
0808 07:00:53.334743 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:53.334754 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:53.334806 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:53.334964 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:53.334964 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:53.335132 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:53.335183 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.335184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.335808 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.335809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.336324 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:53.336325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.336635 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.336636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:53.336681 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.337218 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:53.337227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:53.339387 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
W20260808 07:00:53.339552 6126 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:53.334678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:37439 (local address 127.24.181.126:45549)
0808 07:00:53.334778 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:53.334781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:53.334795 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:00:53.334871 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:53.334872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:53.334883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:53.334957 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:53.335247 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.335741 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.336395 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:53.336572 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:53.336623 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:53.339481 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:37439: BlockingRecv error: recv got EOF from 127.24.181.67:37439 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:54.080129 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:54.075239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:37201)
0808 07:00:54.075373 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:54.075385 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:54.075449 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:54.075656 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:54.075657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:54.075833 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:54.075883 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.075885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.076589 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.076590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.077169 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.077171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.077572 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.077573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.077640 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.078107 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:54.078112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:54.080031 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":102,"threads_started":1}
W20260808 07:00:54.080183 6131 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:54.075346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:37201 (local address 127.24.181.126:45549)
0808 07:00:54.075507 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:54.075509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:54.075517 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:00:54.075542 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:54.075542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:54.075551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:54.075620 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:54.075969 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.076486 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.077270 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.077487 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.077572 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.080109 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:37201: BlockingRecv error: recv got EOF from 127.24.181.65:37201 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:54.211320 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:54.206807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:47287)
0808 07:00:54.206997 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:54.207008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:54.207065 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:54.207178 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:54.207179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:54.207363 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:54.207414 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.207416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.208062 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.208063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.208623 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.208624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.208977 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.208978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.209018 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.209466 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:54.209471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:54.211264 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
W20260808 07:00:54.211467 6131 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:54.206853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:47287 (local address 127.24.181.126:45549)
0808 07:00:54.206901 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:54.206903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:54.207066 (+ 163us) server_negotiation.cc:407] Connection header received
0808 07:00:54.207112 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:54.207113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:54.207123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:54.207174 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:54.207457 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.207989 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.208698 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.208911 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.208965 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.211381 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:47287: BlockingRecv error: recv got EOF from 127.24.181.66:47287 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:54 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:54.344409 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:54.339978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:47539)
0808 07:00:54.340132 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:54.340144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:54.340204 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:54.340320 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:54.340321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:54.340487 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:54.340594 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.340597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.341232 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.341233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.341745 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:54.341747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.342071 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.342072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:54.342113 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.342570 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:54.342574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:54.344351 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260808 07:00:54.344513 6131 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:54.340031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:47539 (local address 127.24.181.126:45549)
0808 07:00:54.340079 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:54.340081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:54.340210 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:00:54.340250 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:54.340251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:54.340262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:54.340303 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:54.340678 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.341163 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.341817 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:54.342007 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:54.342069 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:54.344441 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:47539: BlockingRecv error: recv got EOF from 127.24.181.67:47539 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:55.089363 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:55.080683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:58829)
0808 07:00:55.080863 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:55.080873 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:55.080926 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:55.081071 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:55.081072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:55.081233 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:55.081285 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.081286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.081919 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.081919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.082514 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.082517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.082876 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.082877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.082971 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.087420 (+ 4449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:55.087426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:55.089299 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
W20260808 07:00:55.089488 6138 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:55.080786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:58829 (local address 127.24.181.126:45549)
0808 07:00:55.080901 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:55.080903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:55.080912 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:55.080984 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:55.080984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:55.080994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:55.081056 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:55.081356 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.081857 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.082604 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.082808 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.082864 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.089400 (+ 6536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:58829: BlockingRecv error: recv got EOF from 127.24.181.65:58829 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:55.216928 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:55.211783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:42167)
0808 07:00:55.211982 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:55.211996 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:55.212076 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:55.212249 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:55.212250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:55.212442 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:55.212500 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.212502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.213194 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.213196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.214087 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.214090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.214442 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.214443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.214494 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.215079 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:55.215091 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:55.216868 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
W20260808 07:00:55.217033 6138 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:55.211956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:42167 (local address 127.24.181.126:45549)
0808 07:00:55.212024 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:55.212026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:55.212058 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:00:55.212138 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:55.212138 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:55.212148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:55.212197 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:55.212572 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.213081 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.214160 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.214351 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.214408 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.216963 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:42167: BlockingRecv error: recv got EOF from 127.24.181.66:42167 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:55 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:55.349665 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:55.344950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:35341)
0808 07:00:55.345140 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:55.345149 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:55.345222 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:55.345393 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:55.345394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:55.345579 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:55.345628 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.345629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.346304 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.346305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.346844 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:55.346846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.347228 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.347229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:55.347298 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.347768 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:55.347773 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:55.349566 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
W20260808 07:00:55.349732 6138 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:55.345007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:35341 (local address 127.24.181.126:45549)
0808 07:00:55.345081 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:55.345083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:55.345204 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:00:55.345287 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:55.345288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:55.345303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:55.345385 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:55.345707 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.346214 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.346952 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:55.347150 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:55.347230 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:55.349669 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:35341: BlockingRecv error: recv got EOF from 127.24.181.67:35341 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:56.094691 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:56.089916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:45241)
0808 07:00:56.090090 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:56.090100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:56.090151 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:56.090280 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:56.090280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:56.090453 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:56.090504 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.090505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.091210 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.091211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.091764 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.091766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.092130 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.092131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.092172 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.092663 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:56.092667 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:56.094631 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
W20260808 07:00:56.094842 6145 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:56.090004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:45241 (local address 127.24.181.126:45549)
0808 07:00:56.090149 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:56.090152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:56.090162 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:56.090188 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:56.090189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:56.090198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:56.090268 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:56.090605 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.091128 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.091847 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.092057 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.092118 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.094730 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:45241: BlockingRecv error: recv got EOF from 127.24.181.65:45241 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:56.222020 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:56.217464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:42233)
0808 07:00:56.217638 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:56.217650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:56.217705 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:56.217844 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:56.217844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:56.218017 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:56.218067 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.218069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.218743 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.218744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.219328 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.219330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.219656 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.219657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.219699 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.220161 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:56.220165 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:56.221963 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
W20260808 07:00:56.222249 6145 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:56.217586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:42233 (local address 127.24.181.126:45549)
0808 07:00:56.217708 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:56.217710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:56.217738 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:00:56.217758 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:56.217759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:56.217769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:56.217810 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:56.218140 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.218638 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.219401 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.219586 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.219643 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.222189 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:42233: BlockingRecv error: recv got EOF from 127.24.181.66:42233 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:56 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:56.355147 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:56.350236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:36185)
0808 07:00:56.350410 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:56.350421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:56.350487 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:56.350604 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:56.350604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:56.350807 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:56.350856 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.350858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.351564 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.351565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.352250 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:56.352253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.352572 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.352572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:56.352610 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.353167 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:56.353172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:56.355052 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
W20260808 07:00:56.355181 6145 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:56.350311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:36185 (local address 127.24.181.126:45549)
0808 07:00:56.350382 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:56.350384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:56.350486 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:00:56.350531 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:56.350531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:56.350542 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:56.350595 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:56.350954 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.351445 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.352330 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:56.352508 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:56.352585 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:56.355122 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:36185: BlockingRecv error: recv got EOF from 127.24.181.67:36185 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:57.100188 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:57.095190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:39735)
0808 07:00:57.095376 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:57.095386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:57.095439 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:57.095662 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:57.095663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:57.095836 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:57.095889 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.095890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.096636 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.096637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.097149 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.097150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.097549 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.097550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.097592 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.098127 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:57.098134 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:57.100130 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
W20260808 07:00:57.100340 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:57.095278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:39735 (local address 127.24.181.126:45549)
0808 07:00:57.095524 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:57.095527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:57.095537 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:57.095560 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:57.095560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:57.095570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:57.095644 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:57.096001 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.096563 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.097250 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.097469 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.097536 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.100242 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:39735: BlockingRecv error: recv got EOF from 127.24.181.65:39735 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":184,"threads_started":1}
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:57.227062 6153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:57.222535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:46223)
0808 07:00:57.222738 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:57.222750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:57.222804 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:57.222941 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:57.222942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:57.223122 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:57.223179 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.223181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.223843 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.223844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.224370 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.224372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.224705 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.224706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.224747 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.225197 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:57.225201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:57.226999 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":119,"threads_started":1}
W20260808 07:00:57.227164 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:57.222742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:46223 (local address 127.24.181.126:45549)
0808 07:00:57.222818 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:57.222823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:57.222833 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:57.222857 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:57.222857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:57.222867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:57.222937 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:57.223258 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.223766 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.224446 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.224634 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.224686 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.227096 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:46223: BlockingRecv error: recv got EOF from 127.24.181.66:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:57 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:57.360389 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:57.355641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:51771)
0808 07:00:57.355851 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:57.355860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:57.355935 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:57.356108 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:57.356109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:57.356299 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:57.356347 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.356349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.357034 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.357035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.357555 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:57.357557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.357930 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.357931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:57.357998 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.358470 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:57.358475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:57.360332 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":115,"threads_started":1}
W20260808 07:00:57.360505 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:57.355709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:51771 (local address 127.24.181.126:45549)
0808 07:00:57.355800 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:57.355803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:57.355925 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:00:57.356009 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:57.356009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:57.356023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:57.356099 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:57.356435 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.356945 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.357644 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:57.357841 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:57.357927 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:57.360428 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:51771: BlockingRecv error: recv got EOF from 127.24.181.67:51771 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:58.105515 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:58.100703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:47771)
0808 07:00:58.100869 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:58.100882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:58.100943 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:58.101078 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:58.101078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:58.101290 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:58.101357 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.101359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.102063 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.102064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.102592 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.102593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.103010 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.103011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.103062 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.103553 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:58.103557 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:58.105415 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
W20260808 07:00:58.105626 6157 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:58.100812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:47771 (local address 127.24.181.126:45549)
0808 07:00:58.100941 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:58.100943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:58.100952 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:00:58.100993 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:58.100993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:58.101003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:58.101064 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:58.101415 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.101993 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.102668 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.102929 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.102985 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.105502 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:47771: BlockingRecv error: recv got EOF from 127.24.181.65:47771 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:58.232498 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:58.227721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:38615)
0808 07:00:58.227888 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:58.227899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:58.227981 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:58.228098 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:58.228099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:58.228273 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:58.228331 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.228333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.229248 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.229249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.229773 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.229775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.230118 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.230119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.230170 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.230617 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:58.230622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:58.232440 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260808 07:00:58.232604 6157 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:58.227785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:38615 (local address 127.24.181.126:45549)
0808 07:00:58.227890 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:58.227892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:58.227966 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:00:58.228040 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:58.228041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:58.228051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:58.228092 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:58.228392 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.229175 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.229852 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.230027 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.230078 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.232528 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:38615: BlockingRecv error: recv got EOF from 127.24.181.66:38615 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:58 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:58.365592 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:58.360942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:37071)
0808 07:00:58.361117 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:58.361128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:58.361191 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:58.361326 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:58.361326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:58.361495 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:58.361546 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.361548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.362164 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.362165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.362688 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:58.362689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.363098 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.363099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:58.363146 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.363658 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:58.363662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:58.365532 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
W20260808 07:00:58.365717 6157 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:58.361005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:37071 (local address 127.24.181.126:45549)
0808 07:00:58.361070 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:58.361073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:58.361190 (+ 117us) server_negotiation.cc:407] Connection header received
0808 07:00:58.361246 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:58.361247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:58.361259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:58.361308 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:58.361606 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.362102 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.362765 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:58.363026 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:58.363083 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:58.365628 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:37071: BlockingRecv error: recv got EOF from 127.24.181.67:37071 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:59.110803 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:59.106028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:47843)
0808 07:00:59.106195 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:59.106206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:59.106300 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:59.106469 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:59.106469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:59.106660 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:59.106727 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.106729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.107458 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.107459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.107992 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.107994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.108327 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.108328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.108369 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.108908 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:59.108914 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:59.110744 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
W20260808 07:00:59.110913 6162 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:59.106112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:47843 (local address 127.24.181.126:45549)
0808 07:00:59.106281 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:59.106283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:59.106293 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:00:59.106371 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:59.106372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:59.106382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:59.106450 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:59.106796 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.107371 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.108062 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.108250 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.108315 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.110835 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:47843: BlockingRecv error: recv got EOF from 127.24.181.65:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:59.237869 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:59.233113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:58541)
0808 07:00:59.233294 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:59.233304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:59.233362 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:59.233497 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:59.233497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:59.233670 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:59.233723 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.233725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.234425 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.234426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.235031 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.235033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.235418 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.235419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.235459 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.235985 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:59.235990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:59.237798 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
W20260808 07:00:59.238029 6162 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:59.233197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:58541 (local address 127.24.181.126:45549)
0808 07:00:59.233280 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:59.233282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:59.233367 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:00:59.233403 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:59.233404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:59.233415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:59.233478 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:59.233793 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.234353 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.235134 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.235342 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.235415 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.237935 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:58541: BlockingRecv error: recv got EOF from 127.24.181.66:58541 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:00:59 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:00:59.370767 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:00:59.366116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:57983)
0808 07:00:59.366250 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:00:59.366265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:00:59.366324 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:00:59.366454 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:00:59.366455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:00:59.366628 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:00:59.366678 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.366680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.367422 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.367423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.367949 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:00:59.367950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.368262 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.368263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:00:59.368309 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.368752 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:00:59.368757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:00:59.370700 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":100,"threads_started":1}
W20260808 07:00:59.370887 6162 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:00:59.366210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:57983 (local address 127.24.181.126:45549)
0808 07:00:59.366283 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:00:59.366285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:00:59.366311 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:00:59.366368 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:00:59.366369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:00:59.366380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:00:59.366438 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:00:59.366752 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.367354 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.368021 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:00:59.368199 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:00:59.368247 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:00:59.370801 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:57983: BlockingRecv error: recv got EOF from 127.24.181.67:57983 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:00.120914 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:00.111340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:39503)
0808 07:01:00.111476 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:00.111492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:00.111543 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:00.111713 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:00.111713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:00.111881 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:00.111932 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.111934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.112907 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.112908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.113450 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.113452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.113836 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.113837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.113877 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.118913 (+ 5036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:00.118940 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:00.120841 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
W20260808 07:01:00.121083 6169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:00.111433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:39503 (local address 127.24.181.126:45549)
0808 07:01:00.111575 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:00.111577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:00.111586 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:00.111608 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:00.111609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:00.111619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:00.111692 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:00.112051 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.112832 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.113530 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.113766 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.113823 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.120975 (+ 7152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:39503: BlockingRecv error: recv got EOF from 127.24.181.65:39503 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:00.243592 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:00.238425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:43313)
0808 07:01:00.238575 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:00.238586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:00.238644 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:00.238766 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:00.238767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:00.239005 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:00.239058 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.239060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.239682 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.239682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.240241 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.240242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.240573 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.240574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.240613 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.241075 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:00.241080 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:00.243478 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260808 07:01:00.243634 6169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:00.238534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43313 (local address 127.24.181.126:45549)
0808 07:01:00.238581 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:00.238583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:00.238633 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:01:00.238707 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:00.238708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:00.238717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:00.238754 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:00.239124 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.239605 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.240325 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.240509 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.240561 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.243568 (+ 3007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:43313: BlockingRecv error: recv got EOF from 127.24.181.66:43313 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:00 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:00.376497 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:00.371495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:41747)
0808 07:01:00.371642 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:00.371709 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:00.371809 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:00.371975 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:00.371977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:00.372258 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:00.372312 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.372315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.372964 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.372965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.373557 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:00.373558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.373908 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.373909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:00.373959 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.374456 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:00.374462 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:00.376433 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
W20260808 07:01:00.376574 6169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:00.371637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:41747 (local address 127.24.181.126:45549)
0808 07:01:00.371710 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:00.371712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:00.371808 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:01:00.371872 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:00.371873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:00.371887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:00.371951 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:00.372393 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.372886 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.373642 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:00.373815 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:00.373881 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:00.376525 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:41747: BlockingRecv error: recv got EOF from 127.24.181.67:41747 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:01.126116 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:01.121470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:53081)
0808 07:01:01.121628 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:01.121639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:01.121694 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:01.121833 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:01.121833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:01.122018 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:01.122069 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.122071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.122731 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.122732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.123415 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.123417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.123759 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.123760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.123809 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.124261 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:01.124266 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:01.126047 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
W20260808 07:01:01.126243 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:01.121553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:53081 (local address 127.24.181.126:45549)
0808 07:01:01.121666 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:01.121669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:01.121681 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:01:01.121756 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:01.121757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:01.121768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:01.121822 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:01.122136 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.122632 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.123482 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.123674 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.123724 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.126152 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:53081: BlockingRecv error: recv got EOF from 127.24.181.65:53081 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:01.249238 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:01.244143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:52131)
0808 07:01:01.244344 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:01.244355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:01.244415 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:01.244532 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:01.244532 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:01.244711 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:01.244761 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.244762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.245397 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.245398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.245987 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.245989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.246357 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.246358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.246400 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.246993 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:01.246998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:01.249165 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":125,"threads_started":1}
W20260808 07:01:01.249343 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:01.244243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:52131 (local address 127.24.181.126:45549)
0808 07:01:01.244325 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:01.244327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:01.244434 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:01:01.244463 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:01.244464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:01.244475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:01.244526 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:01.244820 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.245325 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.246074 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.246284 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.246342 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.249275 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:52131: BlockingRecv error: recv got EOF from 127.24.181.66:52131 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:01 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:01.381726 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:01.377090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:59181)
0808 07:01:01.377235 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:01.377253 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:01.377313 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:01.377460 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:01.377461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:01.377637 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:01.377687 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.377688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.378343 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.378343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.378997 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:01.379001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.379374 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.379375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:01.379418 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.379896 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:01.379901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:01.381654 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260808 07:01:01.381861 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:01.377164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:59181 (local address 127.24.181.126:45549)
0808 07:01:01.377247 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:01.377249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:01.377300 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:01:01.377374 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:01.377375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:01.377384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:01.377439 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:01.377756 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.378260 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.379091 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:01.379308 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:01.379363 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:01.381776 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:59181: BlockingRecv error: recv got EOF from 127.24.181.67:59181 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:02.132347 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:02.126805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:51723)
0808 07:01:02.127072 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:02.127087 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:02.127165 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:02.127321 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:02.127322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:02.127579 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:02.127733 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.127737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.128617 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.128618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.129262 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.129264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.129614 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.129615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.129710 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.130396 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:02.130402 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:02.132289 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":223,"threads_started":1}
W20260808 07:01:02.132511 6180 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:02.126944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:51723 (local address 127.24.181.126:45549)
0808 07:01:02.127107 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:02.127109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:02.127164 (+ 55us) server_negotiation.cc:407] Connection header received
0808 07:01:02.127216 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:02.127217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:02.127228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:02.127297 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:02.127844 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.128528 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.129357 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.129547 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.129597 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.132413 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:51723: BlockingRecv error: recv got EOF from 127.24.181.65:51723 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:02.254683 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:02.249838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:51073)
0808 07:01:02.250033 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:02.250046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:02.250131 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:02.250290 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:02.250291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:02.250493 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:02.250541 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.250542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.251319 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.251321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.251887 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.251889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.252269 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.252270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.252331 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.252786 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:02.252792 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:02.254624 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
W20260808 07:01:02.254839 6180 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:02.249930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:51073 (local address 127.24.181.126:45549)
0808 07:01:02.250019 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:02.250022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:02.250109 (+ 87us) server_negotiation.cc:407] Connection header received
0808 07:01:02.250218 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:02.250219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:02.250231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:02.250286 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:02.250618 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.251233 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.251980 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.252181 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.252269 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.254754 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:51073: BlockingRecv error: recv got EOF from 127.24.181.66:51073 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:02 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:02.386708 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:02.382235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:37175)
0808 07:01:02.382361 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:02.382380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:02.382446 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:02.382558 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:02.382558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:02.382742 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:02.382791 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.382793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.383430 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.383431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.384001 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:02.384003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.384331 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.384332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:02.384372 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.384847 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:02.384852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:02.386646 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
W20260808 07:01:02.386857 6180 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:02.382301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:37175 (local address 127.24.181.126:45549)
0808 07:01:02.382364 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:02.382367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:02.382446 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:01:02.382489 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:02.382490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:02.382501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:02.382552 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:02.382856 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.383358 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.384087 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:02.384264 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:02.384318 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:02.386765 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:37175: BlockingRecv error: recv got EOF from 127.24.181.67:37175 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:03.137825 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:03.132968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:55007)
0808 07:01:03.133135 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:03.133171 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:03.133253 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:03.133431 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:03.133432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:03.133636 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:03.133688 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.133690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.134430 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.134431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.135014 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.135015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.135404 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.135406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.135472 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.135936 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:03.135941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:03.137751 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
W20260808 07:01:03.137960 6186 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:03.133068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:55007 (local address 127.24.181.126:45549)
0808 07:01:03.133242 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:03.133244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:03.133254 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:03.133308 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:03.133308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:03.133318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:03.133381 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:03.133807 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.134330 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.135105 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.135315 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.135394 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.137867 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:55007: BlockingRecv error: recv got EOF from 127.24.181.65:55007 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:03.259881 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:03.255210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:35579)
0808 07:01:03.255371 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:03.255384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:03.255447 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:03.255577 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:03.255578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:03.255756 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:03.255805 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.255807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.256468 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.256468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.257016 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.257017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.257373 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.257374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.257415 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.257953 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:03.257958 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:03.259799 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
W20260808 07:01:03.260003 6186 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:03.255351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:35579 (local address 127.24.181.126:45549)
0808 07:01:03.255433 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:03.255435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:03.255444 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:03.255508 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:03.255509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:03.255519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:03.255560 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:03.255889 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.256397 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.257098 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.257306 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.257364 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.259903 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:35579: BlockingRecv error: recv got EOF from 127.24.181.66:35579 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:03 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:03.391999 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:03.387433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:41589)
0808 07:01:03.387606 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:03.387616 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:03.387683 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:03.387813 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:03.387814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:03.387988 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:03.388040 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.388058 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.388681 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.388681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.389244 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:03.389245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.389559 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.389560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:03.389600 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.390059 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:03.390064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:03.391926 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
W20260808 07:01:03.392068 6186 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:03.387473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:41589 (local address 127.24.181.126:45549)
0808 07:01:03.387541 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:03.387548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:03.387683 (+ 135us) server_negotiation.cc:407] Connection header received
0808 07:01:03.387728 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:03.387728 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:03.387740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:03.387793 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:03.388134 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.388614 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.389316 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:03.389495 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:03.389546 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:03.392020 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:41589: BlockingRecv error: recv got EOF from 127.24.181.67:41589 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:04.143030 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:04.138323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:38485)
0808 07:01:04.138470 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:04.138481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:04.138536 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:04.138756 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:04.138757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:04.138974 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:04.139023 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.139025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.139665 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.139666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.140210 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.140211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.140540 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.140540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.140581 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.141065 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:04.141071 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:04.142971 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260808 07:01:04.143175 6190 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:04.138438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38485 (local address 127.24.181.126:45549)
0808 07:01:04.138606 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:04.138609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:04.138624 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:01:04.138645 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:04.138646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:04.138655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:04.138714 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:04.139093 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.139599 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.140284 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.140477 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.140527 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.143082 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:38485: BlockingRecv error: recv got EOF from 127.24.181.65:38485 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:04.265228 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:04.260353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:60789)
0808 07:01:04.260516 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:04.260526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:04.260581 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:04.260728 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:04.260729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:04.260920 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:04.260969 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.260971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.261688 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.261689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.262332 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.262334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.262697 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.262698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.262758 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.263330 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:04.263336 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:04.265127 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
W20260808 07:01:04.265277 6190 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:04.260444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60789 (local address 127.24.181.126:45549)
0808 07:01:04.260516 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:04.260518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:04.260580 (+ 62us) server_negotiation.cc:407] Connection header received
0808 07:01:04.260654 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:04.260654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:04.260666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:04.260712 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:04.261064 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.261589 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.262424 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.262622 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.262699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.265208 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:60789: BlockingRecv error: recv got EOF from 127.24.181.66:60789 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:04 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.67@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:04.397116 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:04.392512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.67:43859)
0808 07:01:04.392749 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:04.392769 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:04.392835 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:04.392966 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:04.392967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:04.393142 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:04.393193 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.393195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.393867 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.393868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.394402 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:04.394403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.394724 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.394725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:04.394765 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.395235 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:04.395239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:04.397055 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":148,"threads_started":1}
W20260808 07:01:04.397219 6190 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:04.392710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:43859 (local address 127.24.181.126:45549)
0808 07:01:04.392784 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:04.392786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:04.392819 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:01:04.392888 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:04.392889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:04.392900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:04.392942 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:04.393280 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.393794 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.394478 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:04.394658 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:04.394710 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:04.397147 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.67:43859: BlockingRecv error: recv got EOF from 127.24.181.67:43859 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 07:01:05 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:05 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.65@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:05.151180 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.143576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.65:40807)
0808 07:01:05.143786 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:05.143796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:05.143856 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:05.143988 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:05.143989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:05.144167 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:05.144222 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.144224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.144863 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.144864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.145390 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.145391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.145739 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.145740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.145780 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.148978 (+ 3198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:05.148983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:05.151121 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":112,"threads_started":1}
W20260808 07:01:05.151298 6195 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:05.143630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:40807 (local address 127.24.181.126:45549)
0808 07:01:05.143791 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.143793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.143860 (+ 67us) server_negotiation.cc:407] Connection header received
0808 07:01:05.143900 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.143901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.143911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.143983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.144284 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.144789 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.145463 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.145646 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.145696 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.151223 (+ 5527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.65:40807: BlockingRecv error: recv got EOF from 127.24.181.65:40807 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Aug 08 07:01:05 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:01:05 dist-test-slave-nfb5 krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.181.66@KRBTEST.COM for oryx/127.24.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:01:05.270694 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.265800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:45549 (local address 127.24.181.66:58967)
0808 07:01:05.265947 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:05.265957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:05.266036 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:05.266191 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:05.266192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:05.266390 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:05.266440 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.266442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.267213 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.267214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.267830 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.267833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.268369 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.268370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.268420 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.268888 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:05.268893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:05.270635 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.181.126:45549: Server oryx/127.24.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260808 07:01:05.270802 6195 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:05.265964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:58967 (local address 127.24.181.126:45549)
0808 07:01:05.266048 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.266050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.266059 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:05.266092 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.266093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.266108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.266180 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.266527 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.267120 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.267965 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.268285 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.268366 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.270730 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.181.66:58967: BlockingRecv error: recv got EOF from 127.24.181.66:58967 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:01:05.310711 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5509
I20260808 07:01:05.322827 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5726
I20260808 07:01:05.333185 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5917
I20260808 07:01:05.341718 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 5406
2026-08-08T07:01:05Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17757 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T07:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:05Z Disabled control of system clock
I20260808 07:01:05.374536 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36087
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:36087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:01:05.456565 6207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:05.456772 6207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:05.456794 6207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:05.457878 6207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:05.457914 6207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:05.457927 6207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:05.457940 6207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:05.457953 6207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:05.459339 6207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:36087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:05.459528 6207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:05.459766 6207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:05.464548 6219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.464589 6215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.464567 6214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.464802 6207 server_base.cc:1034] running on GCE node
I20260808 07:01:05.465078 6207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:05.465370 6207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:05.466549 6207 hybrid_clock.cc:648] HybridClock initialized: now 1786172465466533 us; error 53 us; skew 500 ppm
I20260808 07:01:05.467887 6207 webserver.cc:460] Webserver started at http://127.24.181.126:45473/ using document root <none> and password file <none>
I20260808 07:01:05.468130 6207 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:05.468201 6207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:05.468364 6207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:05.469450 6207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "d65e8387c1964ad3a966413ccb5f0b96"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "1019e5eb4995998fdebceb4d812fb322"
server_key_iv: "e9a66eb59dd4f2fb42a1daeec54314cf"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.469831 6207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "d65e8387c1964ad3a966413ccb5f0b96"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "1019e5eb4995998fdebceb4d812fb322"
server_key_iv: "e9a66eb59dd4f2fb42a1daeec54314cf"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.471257 6207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:05.472036 6226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.472255 6207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:05.472360 6207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "d65e8387c1964ad3a966413ccb5f0b96"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "1019e5eb4995998fdebceb4d812fb322"
server_key_iv: "e9a66eb59dd4f2fb42a1daeec54314cf"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.472445 6207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:05.506729 6207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:05.507179 6207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:05.507354 6207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:05.511487 6207 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:36087
I20260808 07:01:05.511533 6303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:36087 every 8 connection(s)
I20260808 07:01:05.511924 6207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:05.512645 6304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:05.515710 6304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96: Bootstrap starting.
I20260808 07:01:05.516383 6304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:05.516728 6304 log.cc:824] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:05.517550 6304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96: No bootstrap required, opened a new log
I20260808 07:01:05.519397 6304 raft_consensus.cc:348] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } }
I20260808 07:01:05.519567 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 6207
I20260808 07:01:05.519603 6304 raft_consensus.cc:374] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:05.519650 6304 raft_consensus.cc:729] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65e8387c1964ad3a966413ccb5f0b96, State: Initialized, Role: FOLLOWER
I20260808 07:01:05.519670 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:01:05.519831 6304 consensus_queue.cc:260] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } }
I20260808 07:01:05.519897 25301 external_mini_cluster.cc:1428] Setting key 3a33cfc163bfb3a5f496c167ab059908
I20260808 07:01:05.519929 6304 raft_consensus.cc:388] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:05.519972 6304 raft_consensus.cc:482] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:05.520031 6304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:05.520893 6304 raft_consensus.cc:504] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } }
I20260808 07:01:05.520991 6304 leader_election.cc:302] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d65e8387c1964ad3a966413ccb5f0b96; no voters:
I20260808 07:01:05.521201 6304 leader_election.cc:288] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:05.521222 6315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:05.521406 6315 raft_consensus.cc:686] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [term 1 LEADER]: Becoming Leader. State: Replica: d65e8387c1964ad3a966413ccb5f0b96, State: Running, Role: LEADER
I20260808 07:01:05.521490 6304 sys_catalog.cc:564] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:05.521528 6315 consensus_queue.cc:237] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } }
I20260808 07:01:05.521992 6316 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d65e8387c1964ad3a966413ccb5f0b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } } }
I20260808 07:01:05.522076 6316 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:05.522240 6318 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d65e8387c1964ad3a966413ccb5f0b96. Latest consensus state: current_term: 1 leader_uuid: "d65e8387c1964ad3a966413ccb5f0b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65e8387c1964ad3a966413ccb5f0b96" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36087 } } }
I20260808 07:01:05.522312 6318 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:05.522732 6321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:05.523355 6321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:05.525025 6321 catalog_manager.cc:1347] Generated new cluster ID: 3ceec899e63c443cb7cd1dea84892e35
I20260808 07:01:05.525084 6321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:05.527145 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.520475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.24.181.126:36087)
0808 07:01:05.520714 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.520722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.520760 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:01:05.520852 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.520857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.520908 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.521064 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.521436 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.523557 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.524818 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.525406 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.525546 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.526387 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:05.526391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:05.526392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:05.526576 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:05.526611 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:05.526622 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:05.526721 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:05.526796 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:01:05.526883 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260808 07:01:05.533427 6321 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:05.534060 6321 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:05.537156 6321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d65e8387c1964ad3a966413ccb5f0b96: Generated new TSK 0
I20260808 07:01:05.537374 6321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:05.539728 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--builtin_ntp_servers=127.24.181.84:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:01:05.630239 6341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:05.630475 6341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:05.630511 6341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:05.631780 6341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:05.631850 6341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:05.631891 6341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:01:05.633255 6341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:05.633528 6341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:05.633827 6341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:05.639863 6354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.639889 6353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.640038 6341 server_base.cc:1034] running on GCE node
W20260808 07:01:05.640237 6359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.640481 6341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:05.640789 6341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:05.641986 6341 hybrid_clock.cc:648] HybridClock initialized: now 1786172465641962 us; error 55 us; skew 500 ppm
I20260808 07:01:05.643628 6341 webserver.cc:460] Webserver started at http://127.24.181.65:34239/ using document root <none> and password file <none>
I20260808 07:01:05.643922 6341 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:05.643996 6341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:05.644127 6341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:05.645568 6341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "686831990473404db4b34b93dc94dae1"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "5552b8abe5987b73a0725f64c6e46a9d"
server_key_iv: "6ef543767f43f0558553d8e8008936e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.646004 6341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "686831990473404db4b34b93dc94dae1"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "5552b8abe5987b73a0725f64c6e46a9d"
server_key_iv: "6ef543767f43f0558553d8e8008936e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.647737 6341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 07:01:05.648598 6365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.648804 6341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:05.648896 6341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "686831990473404db4b34b93dc94dae1"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "5552b8abe5987b73a0725f64c6e46a9d"
server_key_iv: "6ef543767f43f0558553d8e8008936e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.648963 6341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:05.670509 6341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:05.670833 6341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:05.670972 6341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:05.671294 6341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:05.671650 6341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:05.671686 6341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.671710 6341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:05.671758 6341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.677949 6341 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:41309
I20260808 07:01:05.677999 6529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:41309 every 8 connection(s)
I20260808 07:01:05.678346 6341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:05.683943 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.679034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:39221 (local address 127.24.181.126:36087)
0808 07:01:05.679083 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.679085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.679410 (+ 325us) server_negotiation.cc:407] Connection header received
0808 07:01:05.680027 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.680028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.680044 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.680096 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.680799 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.681324 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.682349 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.682568 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.682649 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.683650 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:05.683651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:05.683652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:05.683659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:05.683674 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:05.683678 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:05.683735 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:05.683823 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:01:05.683860 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:01:05.684168 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.679062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36087 (local address 127.24.181.65:39221)
0808 07:01:05.679264 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:05.679311 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:05.679901 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:05.680167 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:05.680171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:05.680181 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:05.680696 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.680700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.681430 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.681431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.682248 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.682250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.682662 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.682662 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.682812 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.683476 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:05.683477 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:05.683508 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:05.683531 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:05.683743 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:05.683749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:05.683752 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:05.683824 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:01:05.683928 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260808 07:01:05.684715 6530 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36087
I20260808 07:01:05.684818 6530 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:05.685009 6530 heartbeater.cc:511] Master 127.24.181.126:36087 requested a full tablet report, sending...
I20260808 07:01:05.685539 6249 ts_manager.cc:194] Registered new tserver with Master: 686831990473404db4b34b93dc94dae1 (127.24.181.65:41309)
I20260808 07:01:05.686257 6249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.65:39221
I20260808 07:01:05.686399 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 6341
I20260808 07:01:05.686498 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:05.686741 25301 external_mini_cluster.cc:1428] Setting key 7f789281cfb251598a58754eecce40b7
I20260808 07:01:05.687680 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--builtin_ntp_servers=127.24.181.84:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:01:05.769914 6535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:05.770107 6535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:05.770128 6535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:05.771337 6535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:05.771399 6535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:05.771441 6535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:01:05.772861 6535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:05.773108 6535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:05.773417 6535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:05.778345 6545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.778525 6535 server_base.cc:1034] running on GCE node
W20260808 07:01:05.778395 6549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.778350 6544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.778849 6535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:05.779146 6535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:05.780288 6535 hybrid_clock.cc:648] HybridClock initialized: now 1786172465780283 us; error 35 us; skew 500 ppm
I20260808 07:01:05.781589 6535 webserver.cc:460] Webserver started at http://127.24.181.66:39177/ using document root <none> and password file <none>
I20260808 07:01:05.781858 6535 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:05.781931 6535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:05.782083 6535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:05.783198 6535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "c859357599ed4ca2a89e9ea0003a504a"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "294bd3083a00e565fa0b39478126c690"
server_key_iv: "868cceb6fb3e67d8b93687b77c5114bc"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.783565 6535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "c859357599ed4ca2a89e9ea0003a504a"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "294bd3083a00e565fa0b39478126c690"
server_key_iv: "868cceb6fb3e67d8b93687b77c5114bc"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.784889 6535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:05.785647 6556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.785890 6535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:05.786000 6535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "c859357599ed4ca2a89e9ea0003a504a"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "294bd3083a00e565fa0b39478126c690"
server_key_iv: "868cceb6fb3e67d8b93687b77c5114bc"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.786077 6535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:05.811058 6535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:05.811395 6535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:05.811559 6535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:05.811806 6535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:05.812160 6535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:05.812218 6535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.812261 6535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:05.812290 6535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.818171 6535 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:36097
I20260808 07:01:05.818235 6710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:36097 every 8 connection(s)
I20260808 07:01:05.818608 6535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:05.823235 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 6535
I20260808 07:01:05.823356 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:05.823614 25301 external_mini_cluster.cc:1428] Setting key 0361f922102acf4fd021136dab0cecba
I20260808 07:01:05.824438 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.819406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:47139 (local address 127.24.181.126:36087)
0808 07:01:05.819474 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.819476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.819721 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:01:05.820291 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.820292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.820306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.820353 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.821075 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.821662 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.822752 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.823021 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.823114 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.824098 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:05.824100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:05.824100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:05.824105 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:05.824120 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:05.824124 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:05.824175 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:05.824330 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:01:05.824360 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:01:05.824658 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.819390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36087 (local address 127.24.181.66:47139)
0808 07:01:05.819599 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:05.819632 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:05.820149 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:05.820434 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:05.820438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:05.820446 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:05.820972 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.820977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.821765 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.821766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.822662 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.822665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.823131 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.823132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.823266 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.823926 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:05.823926 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:05.823966 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:05.823999 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:05.824247 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:05.824253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:05.824255 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:05.824333 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:01:05.824435 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260808 07:01:05.824657 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--builtin_ntp_servers=127.24.181.84:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:01:05.825168 6711 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36087
I20260808 07:01:05.825284 6711 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:05.825464 6711 heartbeater.cc:511] Master 127.24.181.126:36087 requested a full tablet report, sending...
I20260808 07:01:05.825855 6249 ts_manager.cc:194] Registered new tserver with Master: c859357599ed4ca2a89e9ea0003a504a (127.24.181.66:36097)
I20260808 07:01:05.826347 6249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.66:47139
W20260808 07:01:05.906610 6715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:05.906853 6715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:05.906888 6715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:05.908154 6715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:05.908232 6715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:05.908272 6715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:01:05.909646 6715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:05.909914 6715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:05.910223 6715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:05.915330 6722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.915330 6724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:05.915413 6721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:05.915464 6715 server_base.cc:1034] running on GCE node
I20260808 07:01:05.915819 6715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:05.916067 6715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:05.917217 6715 hybrid_clock.cc:648] HybridClock initialized: now 1786172465917193 us; error 36 us; skew 500 ppm
I20260808 07:01:05.918557 6715 webserver.cc:460] Webserver started at http://127.24.181.67:44221/ using document root <none> and password file <none>
I20260808 07:01:05.918843 6715 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:05.918892 6715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:05.919106 6715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:05.920207 6715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "5cdd0ebbda6347869ff9391d912cbd72"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "d191edcc65e1e16b64ee3479b5604619"
server_key_iv: "66a5faba84e7ae9482a7b149d8305939"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.920572 6715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "5cdd0ebbda6347869ff9391d912cbd72"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "d191edcc65e1e16b64ee3479b5604619"
server_key_iv: "66a5faba84e7ae9482a7b149d8305939"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.922013 6715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:05.922791 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.922968 6715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:05.923046 6715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "5cdd0ebbda6347869ff9391d912cbd72"
format_stamp: "Formatted at 2026-08-08 07:01:05 on dist-test-slave-nfb5"
server_key: "d191edcc65e1e16b64ee3479b5604619"
server_key_iv: "66a5faba84e7ae9482a7b149d8305939"
server_key_version: "encryptionkey@0"
I20260808 07:01:05.923128 6715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:05.942736 6715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:05.943071 6715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:05.943182 6715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:05.943368 6715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:05.943706 6715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:05.943742 6715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.943802 6715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:05.943833 6715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:05.949782 6715 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:43477
I20260808 07:01:05.949851 6902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:43477 every 8 connection(s)
I20260808 07:01:05.950170 6715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:05.955354 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.950884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:34563 (local address 127.24.181.126:36087)
0808 07:01:05.951014 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.951017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.951340 (+ 323us) server_negotiation.cc:407] Connection header received
0808 07:01:05.951816 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.951817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.951828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.951877 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.952604 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.953097 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.954098 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.954297 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.954354 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.955121 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:05.955122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:05.955122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:05.955126 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:05.955141 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:05.955145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:05.955193 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:05.955267 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:01:05.955293 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 07:01:05.955566 6907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:05.950881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36087 (local address 127.24.181.67:34563)
0808 07:01:05.951173 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:05.951232 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:05.951732 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:05.951934 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:05.951938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:05.951946 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:05.952527 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.952533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.953182 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.953183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.954015 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:05.954017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:05.954379 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:05.954379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:05.954482 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:05.954990 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:05.954990 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:05.955018 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:05.955039 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:05.955195 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:05.955197 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:05.955199 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:05.955246 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:01:05.955330 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260808 07:01:05.956069 6903 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36087
I20260808 07:01:05.956175 6903 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:05.956362 6903 heartbeater.cc:511] Master 127.24.181.126:36087 requested a full tablet report, sending...
I20260808 07:01:05.956743 6249 ts_manager.cc:194] Registered new tserver with Master: 5cdd0ebbda6347869ff9391d912cbd72 (127.24.181.67:43477)
I20260808 07:01:05.957101 6249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.67:34563
I20260808 07:01:05.960249 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 6715
I20260808 07:01:05.960345 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:05.960496 25301 external_mini_cluster.cc:1428] Setting key fbbbc7e64fcbcb414ec41e539f4a6c33
I20260808 07:01:05.961364 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:01:05.963192 6314 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:05.962647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.24.181.126:36087)
0808 07:01:05.962754 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:05.962756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:05.962802 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:01:05.962880 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:05.962880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:05.962892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:05.962971 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:05.963130 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51894: BlockingRecv error: recv got EOF from 127.0.0.1:51894 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:01:05.963263 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 6341
I20260808 07:01:05.978909 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 6535
I20260808 07:01:05.985208 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 6715
I20260808 07:01:06.001861 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 6207
2026-08-08T07:01:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (656 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T07:01:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:06Z Disabled control of system clock
I20260808 07:01:06.031353 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43721
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:38489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:01:06.113654 6916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:06.113857 6916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:06.113880 6916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:06.115139 6916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:06.115180 6916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:06.115192 6916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:06.115206 6916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:06.115250 6916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:06.116580 6916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:38489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:06.116835 6916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:06.117122 6916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:06.122491 6926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.122495 6924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.122495 6930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:06.122829 6916 server_base.cc:1034] running on GCE node
I20260808 07:01:06.123086 6916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:06.123385 6916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:06.124554 6916 hybrid_clock.cc:648] HybridClock initialized: now 1786172466124538 us; error 28 us; skew 500 ppm
I20260808 07:01:06.125824 6916 webserver.cc:460] Webserver started at http://127.24.181.126:45381/ using document root <none> and password file <none>
I20260808 07:01:06.126065 6916 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:06.126137 6916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:06.126310 6916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:06.127437 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "10448e6352474c0fa83d348152d3459f"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "2189d04aa9a1f75348a12398663d66e4"
server_key_iv: "21dc514aa89e8cd0e5a5ba82424fd6b7"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.127817 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "10448e6352474c0fa83d348152d3459f"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "2189d04aa9a1f75348a12398663d66e4"
server_key_iv: "21dc514aa89e8cd0e5a5ba82424fd6b7"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.129151 6916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:06.129904 6938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.130090 6916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:06.130191 6916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "10448e6352474c0fa83d348152d3459f"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "2189d04aa9a1f75348a12398663d66e4"
server_key_iv: "21dc514aa89e8cd0e5a5ba82424fd6b7"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.130285 6916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:06.133620 6916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:06.133869 6916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:06.134012 6916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:06.137938 6916 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:43721
I20260808 07:01:06.137988 7005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:43721 every 8 connection(s)
I20260808 07:01:06.138306 6916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:06.139071 7006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:06.141772 7006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: Bootstrap starting.
I20260808 07:01:06.142309 7006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:06.142557 7006 log.cc:824] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:06.143266 7006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: No bootstrap required, opened a new log
I20260808 07:01:06.144829 7006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } }
I20260808 07:01:06.144960 7006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:06.144984 7006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10448e6352474c0fa83d348152d3459f, State: Initialized, Role: FOLLOWER
I20260808 07:01:06.145116 7006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } }
I20260808 07:01:06.145195 7006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:06.145226 7006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:06.145259 7006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:06.145591 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 6916
I20260808 07:01:06.145671 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:01:06.145859 7006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } }
I20260808 07:01:06.145881 25301 external_mini_cluster.cc:1428] Setting key 0ba3fa60838bdd79628b09b24c174cce
I20260808 07:01:06.145979 7006 leader_election.cc:302] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10448e6352474c0fa83d348152d3459f; no voters:
I20260808 07:01:06.146158 7006 leader_election.cc:288] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:06.146190 7009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:06.146384 7009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [term 1 LEADER]: Becoming Leader. State: Replica: 10448e6352474c0fa83d348152d3459f, State: Running, Role: LEADER
I20260808 07:01:06.146449 7006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:06.146508 7009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } }
I20260808 07:01:06.147022 7011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10448e6352474c0fa83d348152d3459f. Latest consensus state: current_term: 1 leader_uuid: "10448e6352474c0fa83d348152d3459f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } } }
I20260808 07:01:06.147189 7011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:06.147224 7010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10448e6352474c0fa83d348152d3459f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10448e6352474c0fa83d348152d3459f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43721 } } }
I20260808 07:01:06.147368 7010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f [sys.catalog]: This master's current role is: LEADER
W20260808 07:01:06.148404 7033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:01:06.148476 7033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:01:06.148576 7022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:06.148753 7022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:06.149395 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.146583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.24.181.126:43721)
0808 07:01:06.147683 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:06.147694 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:06.147745 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:01:06.147868 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:06.147875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:06.147928 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:06.148143 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:06.148890 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:06.148894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:06.148895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:06.148899 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:06.148941 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:06.148957 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:06.149027 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:06.149156 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 07:01:06.149213 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":83,"threads_started":1}
I20260808 07:01:06.150472 7022 catalog_manager.cc:1347] Generated new cluster ID: 4ea93f543f3643b8b83b4e7815c36bcb
I20260808 07:01:06.150527 7022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:06.175596 7022 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:06.175804 7022 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:06.179143 7022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10448e6352474c0fa83d348152d3459f: Generated new TSK 0
I20260808 07:01:06.179322 7022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:06.185292 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--builtin_ntp_servers=127.24.181.84:38489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:01:06.267968 7038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:06.268198 7038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:06.268236 7038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:06.269412 7038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:06.269476 7038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:06.269521 7038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:01:06.271001 7038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:38489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:06.271257 7038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:06.271548 7038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:06.276522 7044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.276566 7045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.276746 7049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:06.276822 7038 server_base.cc:1034] running on GCE node
I20260808 07:01:06.277060 7038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:06.277272 7038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:06.278445 7038 hybrid_clock.cc:648] HybridClock initialized: now 1786172466278435 us; error 40 us; skew 500 ppm
I20260808 07:01:06.279815 7038 webserver.cc:460] Webserver started at http://127.24.181.65:35063/ using document root <none> and password file <none>
I20260808 07:01:06.280056 7038 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:06.280128 7038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:06.280336 7038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:06.281401 7038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8ca383f7f8c24545b9fd3d255a6ec151"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "b7f3b97ad8d35cb22be3de6aec7692a9"
server_key_iv: "7770394feda4ac9cd93f1d298038e3f0"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.281798 7038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8ca383f7f8c24545b9fd3d255a6ec151"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "b7f3b97ad8d35cb22be3de6aec7692a9"
server_key_iv: "7770394feda4ac9cd93f1d298038e3f0"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.283156 7038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:06.283896 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.284073 7038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:06.284185 7038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8ca383f7f8c24545b9fd3d255a6ec151"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "b7f3b97ad8d35cb22be3de6aec7692a9"
server_key_iv: "7770394feda4ac9cd93f1d298038e3f0"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.284263 7038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:06.287588 7038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:06.287839 7038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:06.287992 7038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:06.288208 7038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:06.288506 7038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:06.288559 7038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.288597 7038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:06.288626 7038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.294456 7038 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:38825
I20260808 07:01:06.294544 7216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:38825 every 8 connection(s)
I20260808 07:01:06.294833 7038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:06.297760 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.295585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:48337 (local address 127.24.181.126:43721)
0808 07:01:06.295632 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:06.295634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:06.295875 (+ 241us) server_negotiation.cc:407] Connection header received
0808 07:01:06.296281 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:06.296283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:06.296291 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:06.296363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:06.297544 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:06.297544 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:06.297545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:06.297548 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:06.297559 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:06.297563 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:06.297608 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:06.297693 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:01:06.297707 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:01:06.297917 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.295535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43721 (local address 127.24.181.65:48337)
0808 07:01:06.295727 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:06.295765 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:06.296200 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:06.296432 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:06.296435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:06.296444 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:06.297403 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:06.297404 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:06.297431 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:06.297468 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:06.297618 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:06.297623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:06.297625 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:06.297687 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:01:06.297734 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260808 07:01:06.298480 7218 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43721
I20260808 07:01:06.298590 7218 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:06.298735 7218 heartbeater.cc:511] Master 127.24.181.126:43721 requested a full tablet report, sending...
I20260808 07:01:06.299273 6959 ts_manager.cc:194] Registered new tserver with Master: 8ca383f7f8c24545b9fd3d255a6ec151 (127.24.181.65:38825)
I20260808 07:01:06.300545 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7038
I20260808 07:01:06.300628 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:06.300796 25301 external_mini_cluster.cc:1428] Setting key 9dd99350f2f9769801c9f440c65cb883
I20260808 07:01:06.301617 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--builtin_ntp_servers=127.24.181.84:38489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:01:06.386086 7221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:06.386370 7221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:06.386402 7221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:06.387794 7221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:06.387852 7221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:06.387882 7221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:01:06.389263 7221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:38489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:06.389473 7221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:06.389725 7221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:06.394658 7236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.394670 7228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.394670 7230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:06.394891 7221 server_base.cc:1034] running on GCE node
I20260808 07:01:06.395205 7221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:06.395459 7221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:06.396624 7221 hybrid_clock.cc:648] HybridClock initialized: now 1786172466396604 us; error 39 us; skew 500 ppm
I20260808 07:01:06.397979 7221 webserver.cc:460] Webserver started at http://127.24.181.66:45491/ using document root <none> and password file <none>
I20260808 07:01:06.398238 7221 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:06.398312 7221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:06.398478 7221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:06.399610 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "71bba5a32a0b49ad945bc3fc1b407223"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "fc25ac2b453612557cefac568f806ed3"
server_key_iv: "7f74bfa7929fb7483c580f198bab4e57"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.399981 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "71bba5a32a0b49ad945bc3fc1b407223"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "fc25ac2b453612557cefac568f806ed3"
server_key_iv: "7f74bfa7929fb7483c580f198bab4e57"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.401338 7221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:01:06.402040 7248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.402221 7221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.402334 7221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "71bba5a32a0b49ad945bc3fc1b407223"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "fc25ac2b453612557cefac568f806ed3"
server_key_iv: "7f74bfa7929fb7483c580f198bab4e57"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.402410 7221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:06.406147 7221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:06.406425 7221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:06.406572 7221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:06.406826 7221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:06.407176 7221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:06.407232 7221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.407275 7221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:06.407302 7221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.413196 7221 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:43759
I20260808 07:01:06.413250 7398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:43759 every 8 connection(s)
I20260808 07:01:06.413563 7221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:06.416473 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.414278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37257 (local address 127.24.181.126:43721)
0808 07:01:06.414330 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:06.414332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:06.414547 (+ 215us) server_negotiation.cc:407] Connection header received
0808 07:01:06.415064 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:06.415065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:06.415072 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:06.415125 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:06.416303 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:06.416304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:06.416304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:06.416307 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:06.416318 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:06.416321 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:06.416362 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:06.416408 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:01:06.416419 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:01:06.416692 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.414276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43721 (local address 127.24.181.66:37257)
0808 07:01:06.414447 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:06.414477 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:06.414965 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:06.415178 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:06.415181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:06.415190 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:06.416177 (+ 987us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:06.416177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:06.416204 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:06.416231 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:06.416361 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:06.416363 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:06.416366 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:06.416417 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:01:06.416462 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260808 07:01:06.417003 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7221
I20260808 07:01:06.417114 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:06.417285 7399 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43721
I20260808 07:01:06.417340 25301 external_mini_cluster.cc:1428] Setting key d60f86016f1c387f56c5867ca5aa44f9
I20260808 07:01:06.417382 7399 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:06.417531 7399 heartbeater.cc:511] Master 127.24.181.126:43721 requested a full tablet report, sending...
I20260808 07:01:06.417925 6958 ts_manager.cc:194] Registered new tserver with Master: 71bba5a32a0b49ad945bc3fc1b407223 (127.24.181.66:43759)
I20260808 07:01:06.418345 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--builtin_ntp_servers=127.24.181.84:38489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:01:06.500847 7402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:06.501050 7402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:06.501072 7402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:06.502161 7402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:06.502195 7402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:06.502220 7402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:01:06.503489 7402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:38489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:06.503664 7402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:06.503904 7402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:06.508950 7407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.508950 7410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:06.508947 7408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:06.509351 7402 server_base.cc:1034] running on GCE node
I20260808 07:01:06.509543 7402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:06.509742 7402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:06.510887 7402 hybrid_clock.cc:648] HybridClock initialized: now 1786172466510875 us; error 28 us; skew 500 ppm
I20260808 07:01:06.512219 7402 webserver.cc:460] Webserver started at http://127.24.181.67:46753/ using document root <none> and password file <none>
I20260808 07:01:06.512470 7402 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:06.512518 7402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:06.512666 7402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:06.513818 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "1e28d55456d344119b7087de6858e376"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "8ca287a1fe8f703612b32beda42e2dda"
server_key_iv: "bc5932a7c1737836e1d70d32a965178b"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.514178 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "1e28d55456d344119b7087de6858e376"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "8ca287a1fe8f703612b32beda42e2dda"
server_key_iv: "bc5932a7c1737836e1d70d32a965178b"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.515597 7402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:01:06.516359 7416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.516536 7402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:06.516629 7402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "1e28d55456d344119b7087de6858e376"
format_stamp: "Formatted at 2026-08-08 07:01:06 on dist-test-slave-nfb5"
server_key: "8ca287a1fe8f703612b32beda42e2dda"
server_key_iv: "bc5932a7c1737836e1d70d32a965178b"
server_key_version: "encryptionkey@0"
I20260808 07:01:06.516702 7402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:06.520125 7402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:06.520399 7402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:06.520532 7402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:06.520749 7402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:06.521045 7402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:06.521077 7402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.521128 7402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:06.521159 7402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:06.527034 7402 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41293
I20260808 07:01:06.527087 7576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41293 every 8 connection(s)
I20260808 07:01:06.527390 7402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:06.530309 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.528084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:48987 (local address 127.24.181.126:43721)
0808 07:01:06.528176 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:06.528178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:06.528369 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:01:06.528796 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:06.528797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:06.528803 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:06.528858 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:06.530068 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:06.530070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:06.530070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:06.530075 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:06.530090 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:06.530095 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:06.530148 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:06.530227 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:01:06.530241 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:01:06.530491 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:06.528040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:43721 (local address 127.24.181.67:48987)
0808 07:01:06.528241 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:06.528272 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:06.528708 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:06.528908 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:06.528911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:06.528918 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:06.529910 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:06.529911 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:06.529938 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:06.529966 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:06.530158 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:06.530160 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:06.530163 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:06.530217 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:01:06.530263 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260808 07:01:06.530968 7577 heartbeater.cc:347] Connected to a master server at 127.24.181.126:43721
I20260808 07:01:06.531059 7577 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:06.531172 7577 heartbeater.cc:511] Master 127.24.181.126:43721 requested a full tablet report, sending...
I20260808 07:01:06.531507 6958 ts_manager.cc:194] Registered new tserver with Master: 1e28d55456d344119b7087de6858e376 (127.24.181.67:41293)
I20260808 07:01:06.533707 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7402
I20260808 07:01:06.533823 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:06.534016 25301 external_mini_cluster.cc:1428] Setting key a688ad8bd4a55a1c389901c78e0407f0
I20260808 07:01:06.534968 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:01:06.536677 7013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:06.536243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.24.181.126:43721)
0808 07:01:06.536305 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:06.536306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:06.536387 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:01:06.536437 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:06.536438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:06.536441 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:06.536489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:06.536612 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53676: BlockingRecv error: recv got EOF from 127.0.0.1:53676 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:01:06.536788 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7038
I20260808 07:01:06.547484 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7221
I20260808 07:01:06.556022 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7402
I20260808 07:01:06.564338 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 6916
2026-08-08T07:01:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (565 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:06 dist-test-slave-nfb5 krb5kdc[7592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:06 dist-test-slave-nfb5 krb5kdc[7592](info): set up 2 sockets
Aug 08 07:01:06 dist-test-slave-nfb5 krb5kdc[7592](info): commencing operation
krb5kdc: starting...
W20260808 07:01:07.776120 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:01:07 dist-test-slave-nfb5 krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:01:07.897878 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41289
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:07.979547 7609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:07.979776 7609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:07.979812 7609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:07.980985 7609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:07.981053 7609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:07.981079 7609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:07.981108 7609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:07.981137 7609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:07.982594 7609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:07.982867 7609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:07.983146 7609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:07.988122 7616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:07.988131 7621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:07.988129 7615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:07.988144 7609 server_base.cc:1034] running on GCE node
I20260808 07:01:07.988579 7609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:07.988813 7609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:07.989967 7609 hybrid_clock.cc:648] HybridClock initialized: now 1786172467989954 us; error 30 us; skew 500 ppm
Aug 08 07:01:07 dist-test-slave-nfb5 krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172467, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:07.992406 7609 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:01:07.992869 7609 webserver.cc:460] Webserver started at http://127.24.181.126:35019/ using document root <none> and password file <none>
I20260808 07:01:07.993113 7609 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:07.993181 7609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:07.993332 7609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:07.994374 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "f8792369e27446e98522f177ce2e4f09"
format_stamp: "Formatted at 2026-08-08 07:01:07 on dist-test-slave-nfb5"
server_key: "c5bd1c1b18aa63c4b4b64ae88c087136"
server_key_iv: "ab26cfe88df40964cd1d7b3eb26a0663"
server_key_version: "encryptionkey@0"
I20260808 07:01:07.994725 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "f8792369e27446e98522f177ce2e4f09"
format_stamp: "Formatted at 2026-08-08 07:01:07 on dist-test-slave-nfb5"
server_key: "c5bd1c1b18aa63c4b4b64ae88c087136"
server_key_iv: "ab26cfe88df40964cd1d7b3eb26a0663"
server_key_version: "encryptionkey@0"
I20260808 07:01:07.996130 7609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:07.996861 7631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:07.997041 7609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:07.997141 7609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "f8792369e27446e98522f177ce2e4f09"
format_stamp: "Formatted at 2026-08-08 07:01:07 on dist-test-slave-nfb5"
server_key: "c5bd1c1b18aa63c4b4b64ae88c087136"
server_key_iv: "ab26cfe88df40964cd1d7b3eb26a0663"
server_key_version: "encryptionkey@0"
I20260808 07:01:07.997236 7609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:08.004788 7609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:08.006678 7609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:08.006841 7609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:08.010836 7609 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:41289
I20260808 07:01:08.010885 7714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:41289 every 8 connection(s)
I20260808 07:01:08.011269 7609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:08.012400 7715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:08.012846 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7609
I20260808 07:01:08.012954 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:01:08.013172 25301 external_mini_cluster.cc:1428] Setting key ef973631328049ee9e9c60c2a6225b1c
I20260808 07:01:08.015638 7715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: Bootstrap starting.
I20260808 07:01:08.016256 7715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:08.016543 7715 log.cc:824] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:08.017243 7715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: No bootstrap required, opened a new log
I20260808 07:01:08.019428 7715 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } }
I20260808 07:01:08.019605 7715 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:08.019646 7715 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8792369e27446e98522f177ce2e4f09, State: Initialized, Role: FOLLOWER
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:01:08.019778 7715 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } }
I20260808 07:01:08.019873 7715 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:08.019914 7715 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:08.019971 7715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:08.020656 7715 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } }
I20260808 07:01:08.020799 7715 leader_election.cc:302] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8792369e27446e98522f177ce2e4f09; no voters:
I20260808 07:01:08.020972 7715 leader_election.cc:288] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:08.021018 7723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:08.021183 7723 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [term 1 LEADER]: Becoming Leader. State: Replica: f8792369e27446e98522f177ce2e4f09, State: Running, Role: LEADER
I20260808 07:01:08.021248 7715 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:08.021359 7723 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } }
I20260808 07:01:08.021720 7723 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8792369e27446e98522f177ce2e4f09. Latest consensus state: current_term: 1 leader_uuid: "f8792369e27446e98522f177ce2e4f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } } }
I20260808 07:01:08.021732 7725 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8792369e27446e98522f177ce2e4f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8792369e27446e98522f177ce2e4f09" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41289 } } }
I20260808 07:01:08.021914 7725 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:08.022126 7723 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:08.022286 7740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:08.022850 7740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:08.024400 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.013719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.24.181.126:41289)
0808 07:01:08.013905 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.013913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.013938 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:01:08.014344 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.014350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.014395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.014509 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:08.015379 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.016312 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.017127 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.017593 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.017719 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.020411 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:08.020415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:08.020415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:08.020425 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:08.022609 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.023100 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.023103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.023103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.023200 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.023426 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.023427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.023427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.023735 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:08.023913 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.024019 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.024162 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260808 07:01:08.025039 7740 catalog_manager.cc:1347] Generated new cluster ID: f23480ec62754c26bada89dd64a241d9
I20260808 07:01:08.025089 7740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:08.039067 7740 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:08.039598 7740 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:08.049717 7740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: Generated new TSK 0
I20260808 07:01:08.049973 7740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:01:08.085973 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:08.168465 7757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:08.168659 7757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:08.168682 7757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:08.169747 7757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:08.169781 7757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:08.169807 7757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:01:08.171336 7757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:08.171512 7757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:08.171676 7757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:08.176828 7767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:08.176739 7764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:08.176748 7765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:08.177093 7757 server_base.cc:1034] running on GCE node
I20260808 07:01:08.177274 7757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:08.177479 7757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:08.178617 7757 hybrid_clock.cc:648] HybridClock initialized: now 1786172468178604 us; error 30 us; skew 500 ppm
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:08.181277 7757 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:01:08.181660 7757 webserver.cc:460] Webserver started at http://127.24.181.65:44267/ using document root <none> and password file <none>
I20260808 07:01:08.181857 7757 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:08.181897 7757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:08.181983 7757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:08.183032 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "4c8ab58ec020b863c4d37f7b674dd6b5"
server_key_iv: "13da6532fedb8b3cdee1a63e4161c417"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.183341 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "4c8ab58ec020b863c4d37f7b674dd6b5"
server_key_iv: "13da6532fedb8b3cdee1a63e4161c417"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.184587 7757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:08.185385 7776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.185575 7757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:08.185642 7757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "4c8ab58ec020b863c4d37f7b674dd6b5"
server_key_iv: "13da6532fedb8b3cdee1a63e4161c417"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.185734 7757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:08.200287 7757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:08.202265 7757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:08.202419 7757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:08.202642 7757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:08.202996 7757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:08.203034 7757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.203056 7757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:08.203069 7757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.208847 7757 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:34967
I20260808 07:01:08.208928 7921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:34967 every 8 connection(s)
I20260808 07:01:08.209247 7757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:08.211274 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7757
I20260808 07:01:08.211390 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:08.211606 25301 external_mini_cluster.cc:1428] Setting key 66a09fa4ea0a9249eef955514d67fc9f
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:01:08.219555 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.210039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:40055 (local address 127.24.181.126:41289)
0808 07:01:08.210104 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.210108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.210338 (+ 230us) server_negotiation.cc:407] Connection header received
0808 07:01:08.210858 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.210859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.210867 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.210980 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:08.212155 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.212708 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.213739 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.213963 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.214040 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.216612 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:08.216613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:08.216613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:08.216619 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:08.218000 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.218527 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.218528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.218528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.218579 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.218865 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.218865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.218866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.219016 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:08.219071 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.219421 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.219490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:01:08.219826 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.210017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41289 (local address 127.24.181.65:40055)
0808 07:01:08.210195 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.210244 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.210768 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.211028 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.211031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.211795 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:08.212070 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.212076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.212814 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.212815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.213634 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.213636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.214051 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.214053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.214186 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.214708 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:08.214716 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:08.216521 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:08.218087 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.218095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.218097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.218443 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.218647 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.218648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.218648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.218793 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.219084 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:08.219092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:08.219269 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.219400 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.219575 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260808 07:01:08.220386 7922 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41289
I20260808 07:01:08.220494 7922 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:08.220714 7922 heartbeater.cc:511] Master 127.24.181.126:41289 requested a full tablet report, sending...
I20260808 07:01:08.221287 7654 ts_manager.cc:194] Registered new tserver with Master: c8e8ed002aa34ff7afa175f6fd86e4a3 (127.24.181.65:34967)
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:01:08.221925 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:40055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:01:08.239957 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:08.324275 7931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:08.324517 7931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:08.324541 7931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:08.325661 7931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:08.325700 7931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:08.325726 7931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:01:08.327191 7931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:08.327381 7931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:08.327548 7931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:08.332592 7941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:08.332623 7931 server_base.cc:1034] running on GCE node
W20260808 07:01:08.332605 7945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:08.332638 7939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:08.333142 7931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:08.333393 7931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:08.334559 7931 hybrid_clock.cc:648] HybridClock initialized: now 1786172468334553 us; error 37 us; skew 500 ppm
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:08.337325 7931 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:01:08.337790 7931 webserver.cc:460] Webserver started at http://127.24.181.66:39425/ using document root <none> and password file <none>
I20260808 07:01:08.337997 7931 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:08.338042 7931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:08.338133 7931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:08.339250 7931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fde0b3c6c51761fbf9a51b4c7cfb782d"
server_key_iv: "a2bfa864130022b0d0275f4771c8f015"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.339555 7931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fde0b3c6c51761fbf9a51b4c7cfb782d"
server_key_iv: "a2bfa864130022b0d0275f4771c8f015"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.340876 7931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:08.341584 7955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.341784 7931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:08.341876 7931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fde0b3c6c51761fbf9a51b4c7cfb782d"
server_key_iv: "a2bfa864130022b0d0275f4771c8f015"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.341931 7931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:08.393393 7931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:08.395470 7931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:08.395591 7931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:08.395779 7931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:08.396134 7931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:08.396202 7931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.396245 7931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:08.396275 7931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.402230 7931 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:38589
I20260808 07:01:08.402349 8109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:38589 every 8 connection(s)
I20260808 07:01:08.402652 7931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:08.405539 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 7931
I20260808 07:01:08.405655 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:08.405877 25301 external_mini_cluster.cc:1428] Setting key d7ca99ecef3d4bd1d38f316656d15207
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:01:08.412832 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.403415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:40727 (local address 127.24.181.126:41289)
0808 07:01:08.403482 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.403484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.403728 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:01:08.404225 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.404226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.404237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.404294 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:08.405246 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.405850 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.407341 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.407526 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.407577 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.409861 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:08.409862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:08.409862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:08.409870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:08.411352 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.411793 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.411794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.411794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.411856 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.412116 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.412117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.412117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.412235 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:08.412314 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.412653 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.412733 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:01:08.413045 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.403402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41289 (local address 127.24.181.66:40727)
0808 07:01:08.403589 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.403626 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.404120 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.404378 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.404382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.404954 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:08.405148 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.405153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.405949 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.405950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.407243 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.407246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.407608 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.407609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.407760 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.408256 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:08.408266 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:08.409739 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:08.411455 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.411464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.411466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.411696 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.411927 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.411928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.411928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.412042 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.412305 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:08.412313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:08.412500 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.412631 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.412796 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260808 07:01:08.413547 8111 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41289
I20260808 07:01:08.413647 8111 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:08.413852 8111 heartbeater.cc:511] Master 127.24.181.126:41289 requested a full tablet report, sending...
I20260808 07:01:08.414271 7654 ts_manager.cc:194] Registered new tserver with Master: 5cab39fd3dbd4e0dadaffdd58c0f27f6 (127.24.181.66:38589)
I20260808 07:01:08.414808 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:40727
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:01:08.433847 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:08.517264 8121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:08.517464 8121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:08.517488 8121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:08.518568 8121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:08.518603 8121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:08.518630 8121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:01:08.520089 8121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:08.520290 8121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:08.520463 8121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:08.525939 8126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:08.525899 8131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:08.525975 8121 server_base.cc:1034] running on GCE node
W20260808 07:01:08.525913 8127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:08.526320 8121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:08.526566 8121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:08.527762 8121 hybrid_clock.cc:648] HybridClock initialized: now 1786172468527761 us; error 41 us; skew 500 ppm
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:08.530462 8121 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:01:08.531004 8121 webserver.cc:460] Webserver started at http://127.24.181.67:46255/ using document root <none> and password file <none>
I20260808 07:01:08.531248 8121 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:08.531318 8121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:08.531469 8121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:08.532537 8121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "c509d30af1be47f19dcc973d004ff19e"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fecf0b70c084db1f48d3886ffa7b8ab2"
server_key_iv: "c80a39a6d5c07809189044ce638ce964"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.532915 8121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "c509d30af1be47f19dcc973d004ff19e"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fecf0b70c084db1f48d3886ffa7b8ab2"
server_key_iv: "c80a39a6d5c07809189044ce638ce964"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.534351 8121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:08.535319 8141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.535568 8121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:08.535676 8121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "c509d30af1be47f19dcc973d004ff19e"
format_stamp: "Formatted at 2026-08-08 07:01:08 on dist-test-slave-nfb5"
server_key: "fecf0b70c084db1f48d3886ffa7b8ab2"
server_key_iv: "c80a39a6d5c07809189044ce638ce964"
server_key_version: "encryptionkey@0"
I20260808 07:01:08.535780 8121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:08.544749 8121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:08.547168 8121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:08.547326 8121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:08.547564 8121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:08.547879 8121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:08.547933 8121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.547976 8121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:08.548003 8121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:08.554198 8121 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:34875
I20260808 07:01:08.554262 8292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:34875 every 8 connection(s)
I20260808 07:01:08.554560 8121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:08.558907 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 8121
I20260808 07:01:08.559033 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:08.559242 25301 external_mini_cluster.cc:1428] Setting key d4e5215aeaaef13562f9a245d051a098
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[7592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:01:08.563618 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.555269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:58815 (local address 127.24.181.126:41289)
0808 07:01:08.555332 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.555334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.555550 (+ 216us) server_negotiation.cc:407] Connection header received
0808 07:01:08.556031 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.556032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.556041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.556102 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:08.557005 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.557504 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.558447 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.558639 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.558695 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.560978 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:08.560979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:08.560979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:08.560986 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:08.562430 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.562772 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.562772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.562773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.562824 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.563095 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.563096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.563096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.563196 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:08.563276 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.563497 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.563555 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:01:08.563786 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.555232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41289 (local address 127.24.181.67:58815)
0808 07:01:08.555420 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.555464 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.555948 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.556170 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.556174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.556735 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:08.556924 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.556928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.557586 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.557586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.558366 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.558368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.558733 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.558735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.558899 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.559487 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:08.559495 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:08.560896 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:08.562499 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.562504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.562506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.562704 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.562883 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:08.562884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:08.562884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:08.563037 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:08.563271 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:08.563276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:08.563392 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.563471 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.563588 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260808 07:01:08.564284 8293 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41289
I20260808 07:01:08.564379 8293 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:08.564514 8293 heartbeater.cc:511] Master 127.24.181.126:41289 requested a full tablet report, sending...
I20260808 07:01:08.564888 7654 ts_manager.cc:194] Registered new tserver with Master: c509d30af1be47f19dcc973d004ff19e (127.24.181.67:34875)
I20260808 07:01:08.565256 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:58815
I20260808 07:01:08.571074 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:08.577946 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.572304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.24.181.126:41289)
0808 07:01:08.572348 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.572350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.572511 (+ 161us) server_negotiation.cc:407] Connection header received
0808 07:01:08.572547 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.572547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.572555 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.572601 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:08.573088 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.573562 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.574229 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.574399 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.574449 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.575561 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:08.575562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:08.575562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:08.575568 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:08.576976 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.577276 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.577277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.577277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.577327 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:08.577482 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:08.577483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:08.577483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:08.577607 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:08.577677 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.577763 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.577822 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:01:08.579424 7654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:08.580996 7654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:08.587664 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:38589 (local address 127.0.0.1:57502)
0808 07:01:08.583678 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.583689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.583751 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.583984 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.583986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.584000 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:08.584094 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.584097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585227 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.585227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.586378 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.586381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587349 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.587432 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587448 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.587517 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.587572 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":9,"thread_start_us":80,"threads_started":1}
I20260808 07:01:08.587971 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.24.181.66:38589)
0808 07:01:08.583622 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.583627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.583812 (+ 185us) server_negotiation.cc:407] Connection header received
0808 07:01:08.583859 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.583861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.583877 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.583960 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:08.584184 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585117 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.586476 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587258 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587335 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587392 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.587838 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.587883 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260808 07:01:08.587993 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.24.181.65:34967)
0808 07:01:08.583794 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.583798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.583808 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:08.583858 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.583860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.583877 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.583960 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:08.584371 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585541 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.586825 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587610 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587688 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587866 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.587894 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.587931 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":79,"threads_started":1}
I20260808 07:01:08.588202 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58628 (local address 127.24.181.67:34875)
0808 07:01:08.583774 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.583778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.584307 (+ 529us) server_negotiation.cc:407] Connection header received
0808 07:01:08.584352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.584354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.584370 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.584448 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:08.584567 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585608 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.586807 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587431 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587511 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587566 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.587988 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.588113 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":120,"threads_started":1}
I20260808 07:01:08.587796 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:34875 (local address 127.0.0.1:58628)
0808 07:01:08.584182 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.584193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.584253 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.584446 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.584447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.584449 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:08.584506 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.584508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585717 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.585718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.586719 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.586721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587523 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.587599 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587605 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.587680 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.587729 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":67,"threads_started":1}
I20260808 07:01:08.587921 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.583249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34967 (local address 127.0.0.1:55552)
0808 07:01:08.583548 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.583612 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.583685 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.583982 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.583986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.584003 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:08.584094 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.584096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.585658 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.585659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.586728 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.586731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.587705 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.587706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.587773 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.587778 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.587827 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.587863 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":22,"thread_start_us":52,"threads_started":1}
I20260808 07:01:08.588901 8024 tablet_service.cc:1467] Processing CreateTablet for tablet 70266bb893574a53b39bed2ca77051ca (DEFAULT_TABLE table=test-table [id=ebf5afe803fb4a23b8e9ca2e47800bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:08.588901 8203 tablet_service.cc:1467] Processing CreateTablet for tablet 70266bb893574a53b39bed2ca77051ca (DEFAULT_TABLE table=test-table [id=ebf5afe803fb4a23b8e9ca2e47800bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:08.589298 8024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70266bb893574a53b39bed2ca77051ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:08.589298 8203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70266bb893574a53b39bed2ca77051ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:08.589402 7845 tablet_service.cc:1467] Processing CreateTablet for tablet 70266bb893574a53b39bed2ca77051ca (DEFAULT_TABLE table=test-table [id=ebf5afe803fb4a23b8e9ca2e47800bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:08.589758 7845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70266bb893574a53b39bed2ca77051ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:08.592586 8321 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: Bootstrap starting.
I20260808 07:01:08.593307 8321 tablet_bootstrap.cc:654] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:08.593863 8321 log.cc:824] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:08.594195 8322 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Bootstrap starting.
I20260808 07:01:08.594602 8321 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: No bootstrap required, opened a new log
I20260808 07:01:08.594683 8321 ts_tablet_manager.cc:1397] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:01:08.595484 8322 tablet_bootstrap.cc:654] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:08.595994 8322 log.cc:824] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:08.596349 8323 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Bootstrap starting.
I20260808 07:01:08.596318 8321 raft_consensus.cc:348] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.596495 8321 raft_consensus.cc:374] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:08.596566 8321 raft_consensus.cc:729] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e8ed002aa34ff7afa175f6fd86e4a3, State: Initialized, Role: FOLLOWER
I20260808 07:01:08.596745 8322 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: No bootstrap required, opened a new log
I20260808 07:01:08.596719 8321 consensus_queue.cc:260] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.596824 8322 ts_tablet_manager.cc:1397] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:01:08.596979 8321 ts_tablet_manager.cc:1428] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:08.597112 7922 heartbeater.cc:503] Master 127.24.181.126:41289 was elected leader, sending a full tablet report...
I20260808 07:01:08.598590 8322 raft_consensus.cc:348] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.598732 8322 raft_consensus.cc:374] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:08.598778 8322 raft_consensus.cc:729] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c509d30af1be47f19dcc973d004ff19e, State: Initialized, Role: FOLLOWER
I20260808 07:01:08.598896 8322 consensus_queue.cc:260] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.599264 8322 ts_tablet_manager.cc:1428] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:01:08.599279 8293 heartbeater.cc:503] Master 127.24.181.126:41289 was elected leader, sending a full tablet report...
I20260808 07:01:08.600006 8323 tablet_bootstrap.cc:654] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:08.600603 8323 log.cc:824] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:08.601333 8323 tablet_bootstrap.cc:492] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: No bootstrap required, opened a new log
I20260808 07:01:08.601388 8323 ts_tablet_manager.cc:1397] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:01:08.602782 8323 raft_consensus.cc:348] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.602970 8323 raft_consensus.cc:374] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:08.603024 8323 raft_consensus.cc:729] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cab39fd3dbd4e0dadaffdd58c0f27f6, State: Initialized, Role: FOLLOWER
I20260808 07:01:08.603163 8323 consensus_queue.cc:260] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.603454 8323 ts_tablet_manager.cc:1428] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:01:08.603487 8111 heartbeater.cc:503] Master 127.24.181.126:41289 was elected leader, sending a full tablet report...
W20260808 07:01:08.653244 8112 tablet.cc:2369] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:08.670753 8332 raft_consensus.cc:482] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:08.670890 8332 raft_consensus.cc:504] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.671372 8332 leader_election.cc:288] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8e8ed002aa34ff7afa175f6fd86e4a3 (127.24.181.65:34967), c509d30af1be47f19dcc973d004ff19e (127.24.181.67:34875)
I20260808 07:01:08.674372 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.671577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:34875 (local address 127.24.181.66:34951)
0808 07:01:08.671617 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.671628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.671692 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.671875 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.671876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.671878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:08.671955 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.671957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.672754 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.672755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.673502 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.673504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.674092 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.674093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.674181 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.674184 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.674255 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.674300 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 07:01:08.674372 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.671658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:34951 (local address 127.24.181.67:34875)
0808 07:01:08.671724 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.671726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.671736 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:08.671771 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.671771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.671779 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.671854 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:08.672030 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.672649 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.673606 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.674005 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.674095 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.674105 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.674261 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.674301 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:01:08.674479 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.671578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34967 (local address 127.24.181.66:33231)
0808 07:01:08.671940 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.671951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:08.672036 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:08.672204 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:08.672205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:08.672206 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:08.672291 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.672293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.672909 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.672910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.673627 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:08.673628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.674197 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.674198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:08.674271 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.674277 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:01:08.674351 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:01:08.674392 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":168,"thread_start_us":141,"threads_started":1}
I20260808 07:01:08.674469 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.671661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:33231 (local address 127.24.181.65:34967)
0808 07:01:08.671732 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.671734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.672031 (+ 297us) server_negotiation.cc:407] Connection header received
0808 07:01:08.672096 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.672097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.672110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.672186 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:08.672359 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.672814 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.673726 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.674110 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.674197 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.674205 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.674371 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.674406 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:01:08.674782 8232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70266bb893574a53b39bed2ca77051ca" candidate_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c509d30af1be47f19dcc973d004ff19e" is_pre_election: true
I20260808 07:01:08.674858 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70266bb893574a53b39bed2ca77051ca" candidate_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" is_pre_election: true
I20260808 07:01:08.674991 8232 raft_consensus.cc:2454] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cab39fd3dbd4e0dadaffdd58c0f27f6 in term 0.
I20260808 07:01:08.675034 7868 raft_consensus.cc:2454] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cab39fd3dbd4e0dadaffdd58c0f27f6 in term 0.
I20260808 07:01:08.675252 7957 leader_election.cc:302] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cab39fd3dbd4e0dadaffdd58c0f27f6, c8e8ed002aa34ff7afa175f6fd86e4a3; no voters:
I20260808 07:01:08.675503 8332 raft_consensus.cc:2781] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:08.675575 8332 raft_consensus.cc:482] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:08.675616 8332 raft_consensus.cc:3037] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:08.676339 8332 raft_consensus.cc:504] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.676489 8332 leader_election.cc:288] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [CANDIDATE]: Term 1 election: Requested vote from peers c8e8ed002aa34ff7afa175f6fd86e4a3 (127.24.181.65:34967), c509d30af1be47f19dcc973d004ff19e (127.24.181.67:34875)
I20260808 07:01:08.676666 8232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70266bb893574a53b39bed2ca77051ca" candidate_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c509d30af1be47f19dcc973d004ff19e"
I20260808 07:01:08.676695 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70266bb893574a53b39bed2ca77051ca" candidate_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3"
I20260808 07:01:08.676751 8232 raft_consensus.cc:3037] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:08.676764 7868 raft_consensus.cc:3037] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:08.677688 7868 raft_consensus.cc:2454] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cab39fd3dbd4e0dadaffdd58c0f27f6 in term 1.
I20260808 07:01:08.677688 8232 raft_consensus.cc:2454] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cab39fd3dbd4e0dadaffdd58c0f27f6 in term 1.
I20260808 07:01:08.677872 7957 leader_election.cc:302] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cab39fd3dbd4e0dadaffdd58c0f27f6, c8e8ed002aa34ff7afa175f6fd86e4a3; no voters:
I20260808 07:01:08.678038 8332 raft_consensus.cc:2781] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:08.678202 8332 raft_consensus.cc:686] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 1 LEADER]: Becoming Leader. State: Replica: 5cab39fd3dbd4e0dadaffdd58c0f27f6, State: Running, Role: LEADER
I20260808 07:01:08.678331 8332 consensus_queue.cc:237] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } }
I20260808 07:01:08.679184 7654 catalog_manager.cc:5495] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cab39fd3dbd4e0dadaffdd58c0f27f6 (127.24.181.66). New cstate: current_term: 1 leader_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cab39fd3dbd4e0dadaffdd58c0f27f6" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:08.703356 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.700813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.24.181.66:38589)
0808 07:01:08.700885 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:08.700887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:08.700897 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:08.700950 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:08.700951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:08.700974 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:08.701059 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:08.701216 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.701703 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.702468 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:08.702681 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:08.702758 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:08.702969 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:08.703077 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:08.703166 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:08.703206 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:01:08.703227 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:01:08.705276 7868 raft_consensus.cc:1261] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 1 FOLLOWER]: Refusing update from remote peer 5cab39fd3dbd4e0dadaffdd58c0f27f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:08.705276 8232 raft_consensus.cc:1261] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 1 FOLLOWER]: Refusing update from remote peer 5cab39fd3dbd4e0dadaffdd58c0f27f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:08.705735 8339 consensus_queue.cc:1035] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8e8ed002aa34ff7afa175f6fd86e4a3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:08.705950 8332 consensus_queue.cc:1035] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c509d30af1be47f19dcc973d004ff19e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:08.708882 8345 mvcc.cc:205] Tried to move back new op lower bound from 7316162431813677056 to 7316162431707025408. Current Snapshot: MvccSnapshot[applied={T|T < 7316162431813677056}]
I20260808 07:01:08.710729 7653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:08.710819 7653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:08.711623 7653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8792369e27446e98522f177ce2e4f09: Sending DeleteTablet for 3 replicas of tablet 70266bb893574a53b39bed2ca77051ca
I20260808 07:01:08.711985 8024 tablet_service.cc:1514] Processing DeleteTablet for tablet 70266bb893574a53b39bed2ca77051ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:08 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:57502
I20260808 07:01:08.712025 8203 tablet_service.cc:1514] Processing DeleteTablet for tablet 70266bb893574a53b39bed2ca77051ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:08 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:58628
I20260808 07:01:08.712229 7845 tablet_service.cc:1514] Processing DeleteTablet for tablet 70266bb893574a53b39bed2ca77051ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:08 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:55552
I20260808 07:01:08.712293 8357 tablet_replica.cc:330] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: stopping tablet replica
I20260808 07:01:08.712393 8357 raft_consensus.cc:2229] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:08.712535 8358 tablet_replica.cc:330] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3: stopping tablet replica
I20260808 07:01:08.712538 8357 raft_consensus.cc:2258] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:08.712538 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7757
I20260808 07:01:08.712752 8358 raft_consensus.cc:2229] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:08.712894 8358 raft_consensus.cc:2258] T 70266bb893574a53b39bed2ca77051ca P c8e8ed002aa34ff7afa175f6fd86e4a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:08.713160 8355 tablet_replica.cc:330] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: stopping tablet replica
I20260808 07:01:08.715296 8355 raft_consensus.cc:2229] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:08.715464 8355 raft_consensus.cc:2258] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:08.715991 8355 ts_tablet_manager.cc:1905] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:08.716497 8357 ts_tablet_manager.cc:1905] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:08.717876 8355 ts_tablet_manager.cc:1918] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:08.718034 8355 log.cc:1196] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/70266bb893574a53b39bed2ca77051ca
I20260808 07:01:08.718335 8357 ts_tablet_manager.cc:1918] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:08.718453 8357 log.cc:1196] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/70266bb893574a53b39bed2ca77051ca
I20260808 07:01:08.718468 8355 ts_tablet_manager.cc:1939] T 70266bb893574a53b39bed2ca77051ca P c509d30af1be47f19dcc973d004ff19e: Deleting consensus metadata
I20260808 07:01:08.718775 8357 ts_tablet_manager.cc:1939] T 70266bb893574a53b39bed2ca77051ca P 5cab39fd3dbd4e0dadaffdd58c0f27f6: Deleting consensus metadata
I20260808 07:01:08.718990 7634 catalog_manager.cc:4841] TS c509d30af1be47f19dcc973d004ff19e (127.24.181.67:34875): tablet 70266bb893574a53b39bed2ca77051ca (table test-table [id=ebf5afe803fb4a23b8e9ca2e47800bfd]) successfully deleted
I20260808 07:01:08.719273 7634 catalog_manager.cc:4841] TS 5cab39fd3dbd4e0dadaffdd58c0f27f6 (127.24.181.66:38589): tablet 70266bb893574a53b39bed2ca77051ca (table test-table [id=ebf5afe803fb4a23b8e9ca2e47800bfd]) successfully deleted
W20260808 07:01:08.725698 7632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:34967 (error 108)
I20260808 07:01:08.726264 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:08.726127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34967 (local address 127.0.0.1:55554)
0808 07:01:08.726188 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:08.726236 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:01:08.726421 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7931
W20260808 07:01:08.726461 7632 catalog_manager.cc:4584] TS c8e8ed002aa34ff7afa175f6fd86e4a3 (127.24.181.65:34967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70266bb893574a53b39bed2ca77051ca: Network error: Client connection negotiation failed: client connection to 127.24.181.65:34967: connect: Connection refused (error 111)
I20260808 07:01:08.735944 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 8121
I20260808 07:01:08.746583 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 7609
2026-08-08T07:01:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2180 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[8364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[8364](info): set up 2 sockets
Aug 08 07:01:08 dist-test-slave-nfb5 krb5kdc[8364](info): commencing operation
krb5kdc: starting...
W20260808 07:01:09.978820 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:01:10 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangeradmin_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangeradmin_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangerlookup_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangerlookup_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
I20260808 07:01:10.150090 25301 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNKFpcq/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/postgres -l logfile start
2026-08-08 07:01:12.667 UTC [8403] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:01:12.668 UTC [8403] LOG: listening on IPv4 address "127.24.181.84", port 56765
2026-08-08 07:01:12.671 UTC [8403] LOG: listening on Unix socket "/tmp/.s.PGSQL.56765"
2026-08-08 07:01:12.693 UTC [8410] LOG: database system was shut down at 2026-08-08 07:01:11 UTC
2026-08-08 07:01:12.696 UTC [8403] LOG: database system is ready to accept connections
I20260808 07:01:13.849752 25301 mini_postgres.cc:96] Postgres bound to 56765
2026-08-08 07:01:13.857 UTC [8422] FATAL: database "slave" does not exist
127.24.181.84:56765 - accepting connections
I20260808 07:01:13.858697 25301 mini_ranger.cc:161] Starting Ranger...
I20260808 07:01:13.874485 25301 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:01:14.345902 25301 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:01:14.345999 25301 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin
2026-08-08 07:01:14,675 [I] DB FLAVOR :POSTGRES
2026-08-08 07:01:14,675 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:01:14,676 [I] Checking connection..
2026-08-08 07:01:14,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:01:14,948 [I] Checking connection passed.
2026-08-08 07:01:14,948 [I] --------- Verifying version history table ---------
2026-08-08 07:01:14,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:01:15,219 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:01:15,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:01:15,460 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:01:15,460 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:01:15,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:01:15.710 UTC [8518] WARNING: there is no transaction in progress
2026-08-08 07:01:15,719 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:01:15,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:01:15,990 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:01:15,991 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:01:15,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:01:16,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:01:16,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nfb5.c.gcp-upstream.internal','N') ;"
2026-08-08 07:01:16,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:01:17,005 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:01:17,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:01:17,273 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:01:17,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:01:17,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:01:17,799 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:01:17,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:01:18.752 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.768 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.785 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.802 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.818 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.834 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.889 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.895 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.909 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.918 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.929 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.943 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.953 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.959 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.966 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:18.972 UTC [8661] WARNING: there is no transaction in progress
2026-08-08 07:01:19,590 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:01:19,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:01:19,859 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:01:19,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:01:20,132 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:01:20,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:01:20,393 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:01:20,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:01:20,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:01:20,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nfb5.c.gcp-upstream.internal';"
2026-08-08 07:01:21,171 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:01:21,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:01:21,433 [I] DB_PATCHES have already been applied
I20260808 07:01:21.439836 25301 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNKFpcq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:01:21.439925 25301 mini_ranger.cc:193] Using host: 127.24.181.84
I20260808 07:01:21.443843 25301 mini_ranger.cc:227] Ranger admin URL: http://127.24.181.84:57757
Aug 08, 2026 7:01:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:01:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:01:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:01:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:01:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:01:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:01:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:01:22.853984 25301 mini_ranger.cc:160] Time spent starting Ranger: real 8.995s user 0.000s sys 0.011s
Aug 08, 2026 7:01:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:01:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIs
Aug 08, 2026 7:01:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:01:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:01:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:01:46.442677 25301 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangerkms_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/rangerkms_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.181.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.181.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:01:46 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172506, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:01:46.540141 25301 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:01:46.554628 25301 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:01:47.436618 25301 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:01:47.436691 25301 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms
I20260808 07:01:47.436704 25301 mini_ranger_kms.cc:227] Using postgres at 127.24.181.84:56765
2026-08-08 07:01:48,056 [I] DB FLAVOR :POSTGRES
2026-08-08 07:01:48,057 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:01:48,057 [I] Checking connection
2026-08-08 07:01:48,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:01:48,412 [I] connection success
2026-08-08 07:01:48,412 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:01:48,413 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:01:48,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:01:48,692 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:01:48,692 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:01:48,692 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:01:48,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:56765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:01:48,962 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:01:49.439674 25301 mini_ranger_kms.cc:310] Created kms service
I20260808 07:01:49.612533 25301 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:01:49.658272 25301 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:01:49 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172506, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:01:50.083223 25301 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:01:50.083369 25301 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:01:50.083400 25301 mini_ranger_kms.cc:240] Using host: 127.24.181.84
I20260808 07:01:50.087849 25301 mini_ranger_kms.cc:276] Ranger KMS PID: 8913
I20260808 07:01:50.087972 25301 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.181.84:47021
Aug 08, 2026 7:01:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:01:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:01:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:01:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:01:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:01:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:01:54.017315 25301 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.477s user 0.001s sys 0.560s
I20260808 07:01:54.017536 25301 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:01:54.017715 25301 mini_ranger_kms.cc:401] 127.24.181.84:47021/kms/v1/keys
Aug 08 07:01:54 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:01:55 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
Aug 08 07:01:55 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
Aug 08 07:01:58 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:01:58.914014 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41599
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41599
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:47021/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:59.048207 8975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:59.048446 8975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:59.048472 8975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:59.049896 8975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:59.049940 8975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:59.049957 8975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:59.049971 8975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:59.050024 8975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:59.052052 8975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.181.84:47021/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41599
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:01:59.052328 8975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:59.052651 8975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:59.059315 8985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:59.059289 8989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:59.059255 8986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:59.059582 8975 server_base.cc:1034] running on GCE node
I20260808 07:01:59.059911 8975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:59.060238 8975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:59.061401 8975 hybrid_clock.cc:648] HybridClock initialized: now 1786172519061380 us; error 41 us; skew 500 ppm
Aug 08 07:01:59 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:59.068462 8975 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:01:59.069197 8975 webserver.cc:460] Webserver started at http://127.24.181.126:40945/ using document root <none> and password file <none>
I20260808 07:01:59.069604 8975 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:59.069698 8975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:59.069908 8975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:01:59 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:01:59.135181 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "45669dbc8f6e47a0bdea4bb26600e11d"
format_stamp: "Formatted at 2026-08-08 07:01:59 on dist-test-slave-nfb5"
server_key: "41cddc07d71438356e8a3f80cead89f3"
server_key_iv: "91e16f8c0f538277659d0882a5889e93"
server_key_version: "kuduclusterkey@0"
I20260808 07:01:59.135630 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "45669dbc8f6e47a0bdea4bb26600e11d"
format_stamp: "Formatted at 2026-08-08 07:01:59 on dist-test-slave-nfb5"
server_key: "41cddc07d71438356e8a3f80cead89f3"
server_key_iv: "91e16f8c0f538277659d0882a5889e93"
server_key_version: "kuduclusterkey@0"
I20260808 07:01:59.137305 8975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:01:59.153704 9001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:59.154011 8975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:59.154130 8975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "45669dbc8f6e47a0bdea4bb26600e11d"
format_stamp: "Formatted at 2026-08-08 07:01:59 on dist-test-slave-nfb5"
server_key: "41cddc07d71438356e8a3f80cead89f3"
server_key_iv: "91e16f8c0f538277659d0882a5889e93"
server_key_version: "kuduclusterkey@0"
I20260808 07:01:59.154239 8975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:59.170795 8975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:59.173389 8975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:59.173652 8975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:59.184437 8975 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:41599
I20260808 07:01:59.184469 9060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:41599 every 8 connection(s)
I20260808 07:01:59.185024 8975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:59.185992 9062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:59.189218 9062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: Bootstrap starting.
I20260808 07:01:59.190235 9062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:59.191043 9062 log.cc:824] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:59.191530 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 8975
I20260808 07:01:59.191656 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:01:59.191843 25301 external_mini_cluster.cc:1428] Setting key 6be7f62dfd3e121f44a015aae487a3d9
I20260808 07:01:59.192059 9062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: No bootstrap required, opened a new log
I20260808 07:01:59.196570 9062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } }
I20260808 07:01:59.196750 9062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:59.196787 9062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45669dbc8f6e47a0bdea4bb26600e11d, State: Initialized, Role: FOLLOWER
I20260808 07:01:59.196964 9062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } }
I20260808 07:01:59.197050 9062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:59.197082 9062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:59.197149 9062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:59.198042 9062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } }
I20260808 07:01:59.198164 9062 leader_election.cc:302] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45669dbc8f6e47a0bdea4bb26600e11d; no voters:
I20260808 07:01:59.198395 9062 leader_election.cc:288] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:59.198693 9069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:59.198829 9062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:59.199081 9069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [term 1 LEADER]: Becoming Leader. State: Replica: 45669dbc8f6e47a0bdea4bb26600e11d, State: Running, Role: LEADER
I20260808 07:01:59.199381 9069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } }
I20260808 07:01:59.200029 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } } }
I20260808 07:01:59.200206 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:59.200493 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45669dbc8f6e47a0bdea4bb26600e11d. Latest consensus state: current_term: 1 leader_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45669dbc8f6e47a0bdea4bb26600e11d" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41599 } } }
I20260808 07:01:59.200667 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:59.200899 9087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:01:59 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:01:59.201261 9062 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:01:59.201622 9087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:59.205003 9087 catalog_manager.cc:1347] Generated new cluster ID: ee9adf63b9024b5ab6d8cb4aa44dbe92
I20260808 07:01:59.205076 9087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:59.211825 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:59.192461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.24.181.126:41599)
0808 07:01:59.192730 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:59.192740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:59.192772 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:01:59.194093 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:59.194101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:59.194151 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:59.194314 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:59.195135 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:59.196147 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:59.197151 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:59.197639 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:59.197770 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:59.201831 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:59.201835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:59.201836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:59.201849 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:59.204355 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:59.204796 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:59.204807 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:59.204808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:59.204915 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:59.205155 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:59.205156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:59.205156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:59.205439 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:59.205904 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:59.205976 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:01:59.210995 (+ 5019us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":124,"threads_started":1}
I20260808 07:01:59.231369 9087 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:59.232198 9087 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:59.239457 9087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: Generated new TSK 0
I20260808 07:01:59.239683 9087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:02:00.155952 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:47021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:00.295387 9129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:00.295832 9129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:00.295955 9129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:00.298118 9129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:00.298229 9129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:00.298324 9129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:02:00.300977 9129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.181.84:47021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:00.301457 9129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:00.301785 9129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:00.310963 9136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:00.311018 9135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:00.311079 9140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:00.312080 9129 server_base.cc:1034] running on GCE node
I20260808 07:02:00.312366 9129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:00.312708 9129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:00.313921 9129 hybrid_clock.cc:648] HybridClock initialized: now 1786172520313897 us; error 54 us; skew 500 ppm
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:00.317720 9129 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:02:00.318312 9129 webserver.cc:460] Webserver started at http://127.24.181.65:42343/ using document root <none> and password file <none>
I20260808 07:02:00.318620 9129 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:00.318702 9129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:00.318893 9129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:00.353206 9129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "030c17abe22645e1af5c70ed9847d80a"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "834725e84472f90bd825ad81ee9a3765"
server_key_iv: "cb4b8b80d3ceab72b8beb91a20b2b8e9"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.353698 9129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "030c17abe22645e1af5c70ed9847d80a"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "834725e84472f90bd825ad81ee9a3765"
server_key_iv: "cb4b8b80d3ceab72b8beb91a20b2b8e9"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.358332 9129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 07:02:00.373487 9151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.373750 9129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:00.373836 9129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "030c17abe22645e1af5c70ed9847d80a"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "834725e84472f90bd825ad81ee9a3765"
server_key_iv: "cb4b8b80d3ceab72b8beb91a20b2b8e9"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.373930 9129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:00.409803 9129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:00.412712 9129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:00.412953 9129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:00.413460 9129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:00.413975 9129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:00.414054 9129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.414111 9129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:00.414149 9129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.435160 9129 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36401
I20260808 07:02:00.435894 9129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:00.436468 9306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36401 every 8 connection(s)
I20260808 07:02:00.441493 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 9129
I20260808 07:02:00.441601 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:00.441835 25301 external_mini_cluster.cc:1428] Setting key a96d0fc26e58d321f20f87abc4b01d4f
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:00.446656 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:00.436879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41599 (local address 127.24.181.65:41205)
0808 07:02:00.437163 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:00.437216 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:00.437744 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:00.438027 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:00.438030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:00.438228 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:00.438440 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:00.438444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.439273 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.439275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:00.440072 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:00.440075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.440498 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.440499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:00.440627 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:00.441209 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:00.441218 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:00.442880 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:00.444809 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:00.444818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:00.444821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:00.445153 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:00.445385 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:00.445386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:00.445386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:00.445522 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:00.445839 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:00.445847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:00.446030 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 07:02:00.446148 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:02:00.446317 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":94,"threads_started":1}
I20260808 07:02:00.447335 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:00.437047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:41205 (local address 127.24.181.126:41599)
0808 07:02:00.437164 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:00.437166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:00.437322 (+ 156us) server_negotiation.cc:407] Connection header received
0808 07:02:00.437861 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:00.437862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:00.437878 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:00.437951 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:00.438543 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.439172 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.440168 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.440379 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.440488 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:00.443007 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:00.443009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:00.443009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:00.443017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:00.444673 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:00.445243 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:00.445245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:00.445246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:00.445314 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:00.445593 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:00.445594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:00.445594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:00.445738 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:00.445810 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:00.447077 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0808 07:02:00.447189 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260808 07:02:00.448164 9307 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41599
I20260808 07:02:00.448288 9307 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:00.448490 9307 heartbeater.cc:511] Master 127.24.181.126:41599 requested a full tablet report, sending...
I20260808 07:02:00.449113 9021 ts_manager.cc:194] Registered new tserver with Master: 030c17abe22645e1af5c70ed9847d80a (127.24.181.65:36401)
I20260808 07:02:00.450042 9021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:41205
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:02:00.475447 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:47021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:00.629591 9315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:00.629873 9315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:00.629913 9315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:00.631824 9315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:00.631883 9315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:00.631924 9315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:02:00.634330 9315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.181.84:47021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:00.634588 9315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:00.635363 9315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:00.642598 9332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:00.642611 9326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:00.642719 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:00.642845 9315 server_base.cc:1034] running on GCE node
I20260808 07:02:00.643232 9315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:00.643522 9315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:00.644773 9315 hybrid_clock.cc:648] HybridClock initialized: now 1786172520644747 us; error 38 us; skew 500 ppm
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:00.648885 9315 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:02:00.649420 9315 webserver.cc:460] Webserver started at http://127.24.181.66:43203/ using document root <none> and password file <none>
I20260808 07:02:00.649731 9315 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:00.649804 9315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:00.649948 9315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:00.663460 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "64ec3117585154aaf90840bc41111041"
server_key_iv: "11f0dade210fc08d3d4acaeee6442d89"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.663942 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "64ec3117585154aaf90840bc41111041"
server_key_iv: "11f0dade210fc08d3d4acaeee6442d89"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.668672 9315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260808 07:02:00.692695 9343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.698240 9315 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.000s
I20260808 07:02:00.698398 9315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "64ec3117585154aaf90840bc41111041"
server_key_iv: "11f0dade210fc08d3d4acaeee6442d89"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.698534 9315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:00.724432 9315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:00.727550 9315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:00.727839 9315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:00.728253 9315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:00.728780 9315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:00.728880 9315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.728964 9315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:00.729029 9315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:00.749186 9315 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:43283
I20260808 07:02:00.749400 9508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:43283 every 8 connection(s)
I20260808 07:02:00.749697 9315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:00.755770 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 9315
I20260808 07:02:00.755892 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:00.756086 25301 external_mini_cluster.cc:1428] Setting key 4ec61b3d727b7e80d3226a966b3b3a6b
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:00.764431 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:00.750718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41599 (local address 127.24.181.66:43721)
0808 07:02:00.751701 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:00.751753 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:00.752907 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:00.753244 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:00.753249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:00.753721 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:00.754206 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:00.754213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.755305 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.755307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:00.757014 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:00.757017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.757738 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.757739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:00.757974 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:00.758765 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:00.758776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:00.760806 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:00.762773 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:00.762781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:00.762784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:00.763109 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:00.763336 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:00.763336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:00.763337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:00.763468 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:00.763757 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:00.763763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:00.763931 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 07:02:00.764027 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:02:00.764178 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":165,"threads_started":1}
I20260808 07:02:00.765141 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:00.750673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43721 (local address 127.24.181.126:41599)
0808 07:02:00.750727 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:00.750729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:00.752009 (+ 1280us) server_negotiation.cc:407] Connection header received
0808 07:02:00.753057 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:00.753059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:00.753076 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:00.753151 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:00.754333 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.755189 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.757151 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:00.757570 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:00.757632 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:00.760923 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:00.760925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:00.760925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:00.760933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:00.762676 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:00.763199 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:00.763201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:00.763201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:00.763264 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:00.763539 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:00.763540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:00.763540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:00.763674 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:00.763732 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:00.764956 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0808 07:02:00.765049 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:00.765524 9510 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41599
I20260808 07:02:00.765658 9510 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:00.765883 9510 heartbeater.cc:511] Master 127.24.181.126:41599 requested a full tablet report, sending...
I20260808 07:02:00.766324 9021 ts_manager.cc:194] Registered new tserver with Master: e0c5bb06ed5d47fcab2413aa14f94d4c (127.24.181.66:43283)
I20260808 07:02:00.766700 9021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:43721
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:02:00.796312 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:47021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:00.943164 9520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:00.943470 9520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:00.943509 9520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:00.945335 9520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:00.945396 9520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:00.945441 9520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:02:00.947916 9520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.181.84:47021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:41599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:00.948318 9520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:00.948593 9520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:00.956519 9528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:00.957722 9520 server_base.cc:1034] running on GCE node
W20260808 07:02:00.958010 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:00.959167 9526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:00.959476 9520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:00.959748 9520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:00.967213 9520 hybrid_clock.cc:648] HybridClock initialized: now 1786172520967172 us; error 38 us; skew 500 ppm
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:00.971978 9520 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:02:00.972585 9520 webserver.cc:460] Webserver started at http://127.24.181.67:40167/ using document root <none> and password file <none>
I20260808 07:02:00.972936 9520 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:00.973070 9520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:00.973268 9520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:00 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:00.987612 9520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "5f0d74df3051b6bdf9415730af9dd352"
server_key_iv: "533889e98fda883135e4cf4f174b1e30"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.988174 9520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "5f0d74df3051b6bdf9415730af9dd352"
server_key_iv: "533889e98fda883135e4cf4f174b1e30"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:00.990155 9520 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:01.009989 9544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.010259 9520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 07:02:01.010358 9520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e"
format_stamp: "Formatted at 2026-08-08 07:02:00 on dist-test-slave-nfb5"
server_key: "5f0d74df3051b6bdf9415730af9dd352"
server_key_iv: "533889e98fda883135e4cf4f174b1e30"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:01.010457 9520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:01.032691 9520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:01.035383 9520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:01.035615 9520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:01.036032 9520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:01.036556 9520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:01.036674 9520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.036748 9520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:01.036803 9520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.058481 9520 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41487
I20260808 07:02:01.058516 9716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41487 every 8 connection(s)
I20260808 07:02:01.059216 9520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:01.060299 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 9520
I20260808 07:02:01.060482 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:01.060753 25301 external_mini_cluster.cc:1428] Setting key 75275ef51a7b9c97d36b7d1a85b7f978
Aug 08 07:02:01 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:01.069998 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.060179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:38041 (local address 127.24.181.126:41599)
0808 07:02:01.060232 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.060234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.060946 (+ 712us) server_negotiation.cc:407] Connection header received
0808 07:02:01.061709 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.061711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.061724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.061798 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.062748 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.063413 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.064413 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.064627 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.064706 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.066872 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:01.066873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:01.066874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:01.066882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:01.068519 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.068977 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.068978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.068978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.069045 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.069327 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.069328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.069329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.069448 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:01.069526 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.069809 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.069901 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:02:01.070201 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.060248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41599 (local address 127.24.181.67:38041)
0808 07:02:01.060748 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.060798 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.061594 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.061942 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.061946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.062313 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:01.062647 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.062653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.063507 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.063508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.064313 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.064315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.064714 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.064715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.064864 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.065456 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:01.065464 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:01.066769 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:01.068636 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.068644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.068646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.068882 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.069119 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.069120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.069121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.069245 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.069518 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:01.069526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:01.069672 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.069786 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.069947 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":223,"threads_started":1}
I20260808 07:02:01.070773 9718 heartbeater.cc:347] Connected to a master server at 127.24.181.126:41599
I20260808 07:02:01.070864 9718 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:01.071092 9718 heartbeater.cc:511] Master 127.24.181.126:41599 requested a full tablet report, sending...
I20260808 07:02:01.071523 9021 ts_manager.cc:194] Registered new tserver with Master: ecfe10bd50c245e5a7b99ba319e6fb2e (127.24.181.67:41487)
I20260808 07:02:01.072186 9021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:38041
I20260808 07:02:01.073200 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:01.081830 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.074866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.24.181.126:41599)
0808 07:02:01.074905 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.074907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.075276 (+ 369us) server_negotiation.cc:407] Connection header received
0808 07:02:01.075345 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.075345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.075355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.075411 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.076058 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.076533 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.077263 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.077465 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.077555 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.078774 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:01.078776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:01.078776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:01.078783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:01.080471 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.080997 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.080998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.080998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.081051 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.081301 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.081301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.081302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.081407 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:01.081482 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.081707 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.081765 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:02:01.085503 9021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:01.089561 9021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:01.103178 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.094726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36401 (local address 127.0.0.1:46942)
0808 07:02:01.095484 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.095534 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.095641 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.096209 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.096212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.096215 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:01.096385 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.096397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.098637 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.098638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.100488 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.100502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.102413 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.102414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.102554 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.102570 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.102682 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.102785 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":424,"threads_started":1}
I20260808 07:02:01.103888 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.095332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.24.181.65:36401)
0808 07:02:01.095532 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.095537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.095841 (+ 304us) server_negotiation.cc:407] Connection header received
0808 07:02:01.095907 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.095910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.095939 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.096064 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:01.096546 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.098466 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.100731 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.102157 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.102255 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.102369 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.103681 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.103744 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260808 07:02:01.105080 9215 tablet_service.cc:1467] Processing CreateTablet for tablet 016147409b9c41298e061c459a02fce3 (DEFAULT_TABLE table=test-table [id=861c3ac82ba0470e9be087eaad1c7aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:01.105494 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.094762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43283 (local address 127.0.0.1:58708)
0808 07:02:01.096287 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.096300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.096392 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.096953 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.096954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.096956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:01.097032 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.097033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.099752 (+ 2719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.099754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.101890 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.101894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.105072 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.105073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.105172 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.105176 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.105282 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.105373 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1483,"thread_start_us":686,"threads_started":1}
I20260808 07:02:01.105764 9215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016147409b9c41298e061c459a02fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:01.105891 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.095012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.24.181.66:43283)
0808 07:02:01.095341 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.095347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.096470 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 07:02:01.096565 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.096581 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.096600 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.096726 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:01.097173 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.099568 (+ 2395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.102148 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.104715 (+ 2567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.104833 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.105024 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.105680 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.105731 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":161,"threads_started":1}
I20260808 07:02:01.106998 9417 tablet_service.cc:1467] Processing CreateTablet for tablet 016147409b9c41298e061c459a02fce3 (DEFAULT_TABLE table=test-table [id=861c3ac82ba0470e9be087eaad1c7aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:01.107539 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016147409b9c41298e061c459a02fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:01.110762 9745 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Bootstrap starting.
I20260808 07:02:01.112510 9745 tablet_bootstrap.cc:654] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:01.113322 9745 log.cc:824] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:01.114593 9745 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: No bootstrap required, opened a new log
I20260808 07:02:01.114691 9745 ts_tablet_manager.cc:1397] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:02:01.115288 9747 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Bootstrap starting.
I20260808 07:02:01.116735 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.095636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41487 (local address 127.0.0.1:51356)
0808 07:02:01.103476 (+ 7840us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.103490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.103563 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.111461 (+ 7898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.111463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.111465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:01.111514 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.111516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.113291 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.113293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.114529 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.114532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.116493 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.116494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.116559 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.116563 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.116597 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.116641 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7817,"thread_start_us":195,"threads_started":1}
I20260808 07:02:01.117223 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.099660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.24.181.67:41487)
0808 07:02:01.111103 (+ 11443us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.111109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.111122 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:01.111181 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.111183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.111212 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.111313 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:01.111634 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.113164 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.114809 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.116391 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.116909 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.117064 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.117094 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.117133 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11386,"thread_start_us":110,"threads_started":1}
I20260808 07:02:01.117810 9747 tablet_bootstrap.cc:654] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:01.118057 9618 tablet_service.cc:1467] Processing CreateTablet for tablet 016147409b9c41298e061c459a02fce3 (DEFAULT_TABLE table=test-table [id=861c3ac82ba0470e9be087eaad1c7aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:01.118413 9618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016147409b9c41298e061c459a02fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:01.118495 9745 raft_consensus.cc:348] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.118649 9745 raft_consensus.cc:374] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:01.118680 9745 raft_consensus.cc:729] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 030c17abe22645e1af5c70ed9847d80a, State: Initialized, Role: FOLLOWER
I20260808 07:02:01.118934 9747 log.cc:824] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:01.119511 9745 consensus_queue.cc:260] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.119874 9745 ts_tablet_manager.cc:1428] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260808 07:02:01.120054 9307 heartbeater.cc:503] Master 127.24.181.126:41599 was elected leader, sending a full tablet report...
I20260808 07:02:01.121656 9747 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: No bootstrap required, opened a new log
I20260808 07:02:01.121809 9747 ts_tablet_manager.cc:1397] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 07:02:01.124393 9750 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Bootstrap starting.
I20260808 07:02:01.125578 9747 raft_consensus.cc:348] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.125828 9747 raft_consensus.cc:374] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:01.125869 9750 tablet_bootstrap.cc:654] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:01.125934 9747 raft_consensus.cc:729] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c5bb06ed5d47fcab2413aa14f94d4c, State: Initialized, Role: FOLLOWER
I20260808 07:02:01.126149 9747 consensus_queue.cc:260] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.126480 9747 ts_tablet_manager.cc:1428] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 07:02:01.126679 9510 heartbeater.cc:503] Master 127.24.181.126:41599 was elected leader, sending a full tablet report...
I20260808 07:02:01.127398 9750 log.cc:824] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:01.128479 9750 tablet_bootstrap.cc:492] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: No bootstrap required, opened a new log
I20260808 07:02:01.128574 9750 ts_tablet_manager.cc:1397] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 07:02:01.131435 9750 raft_consensus.cc:348] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.131639 9750 raft_consensus.cc:374] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:01.131706 9750 raft_consensus.cc:729] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe10bd50c245e5a7b99ba319e6fb2e, State: Initialized, Role: FOLLOWER
I20260808 07:02:01.131836 9750 consensus_queue.cc:260] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.132208 9750 ts_tablet_manager.cc:1428] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260808 07:02:01.135025 9718 heartbeater.cc:503] Master 127.24.181.126:41599 was elected leader, sending a full tablet report...
W20260808 07:02:01.187397 9309 tablet.cc:2369] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:01.250710 9511 tablet.cc:2369] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:02:01 dist-test-slave-nfb5 krb5kdc[8364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
W20260808 07:02:01.310101 9719 tablet.cc:2369] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:01.323021 9752 raft_consensus.cc:482] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:01.323201 9752 raft_consensus.cc:504] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.323679 9752 leader_election.cc:288] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 030c17abe22645e1af5c70ed9847d80a (127.24.181.65:36401), ecfe10bd50c245e5a7b99ba319e6fb2e (127.24.181.67:41487)
I20260808 07:02:01.328627 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.324312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37123 (local address 127.24.181.67:41487)
0808 07:02:01.324364 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.324367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.324376 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:01.324411 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.324412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.324422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.324480 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:01.324794 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.325682 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.327478 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.328130 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.328195 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.328205 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.328500 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.328551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:02:01.328647 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.324199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43327 (local address 127.24.181.65:36401)
0808 07:02:01.324254 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.324256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.324426 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:02:01.324538 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.324539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.324552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.324584 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:01.324881 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.325675 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.327478 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.328208 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.328255 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.328266 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.328531 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.328579 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:02:01.328960 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.323879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41487 (local address 127.24.181.66:37123)
0808 07:02:01.324207 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.324221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.324287 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.324539 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.324540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.324542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:01.324671 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.324674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.325852 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.325853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.327310 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.327314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.328280 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.328281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.328393 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.328397 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.328773 (+ 376us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.328842 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":59,"thread_start_us":124,"threads_started":1}
I20260808 07:02:01.329319 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.323880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36401 (local address 127.24.181.66:43327)
0808 07:02:01.324372 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.324382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.324451 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.324659 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.324660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.324661 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:01.324787 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.324789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.325800 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.325801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.327360 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.327362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.328332 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.328332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.328423 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.328427 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.329140 (+ 713us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.329196 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":16,"thread_start_us":119,"threads_started":1}
I20260808 07:02:01.329459 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016147409b9c41298e061c459a02fce3" candidate_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" is_pre_election: true
I20260808 07:02:01.329639 9641 raft_consensus.cc:2454] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0c5bb06ed5d47fcab2413aa14f94d4c in term 0.
I20260808 07:02:01.329717 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016147409b9c41298e061c459a02fce3" candidate_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030c17abe22645e1af5c70ed9847d80a" is_pre_election: true
I20260808 07:02:01.329892 9238 raft_consensus.cc:2454] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0c5bb06ed5d47fcab2413aa14f94d4c in term 0.
I20260808 07:02:01.329934 9347 leader_election.cc:302] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0c5bb06ed5d47fcab2413aa14f94d4c, ecfe10bd50c245e5a7b99ba319e6fb2e; no voters:
I20260808 07:02:01.330075 9752 raft_consensus.cc:2781] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:01.330124 9752 raft_consensus.cc:482] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:01.330150 9752 raft_consensus.cc:3037] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:01.331120 9752 raft_consensus.cc:504] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.331290 9752 leader_election.cc:288] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [CANDIDATE]: Term 1 election: Requested vote from peers 030c17abe22645e1af5c70ed9847d80a (127.24.181.65:36401), ecfe10bd50c245e5a7b99ba319e6fb2e (127.24.181.67:41487)
I20260808 07:02:01.331444 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016147409b9c41298e061c459a02fce3" candidate_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "030c17abe22645e1af5c70ed9847d80a"
I20260808 07:02:01.331488 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016147409b9c41298e061c459a02fce3" candidate_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e"
I20260808 07:02:01.331524 9238 raft_consensus.cc:3037] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:01.331562 9641 raft_consensus.cc:3037] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:01.332711 9238 raft_consensus.cc:2454] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0c5bb06ed5d47fcab2413aa14f94d4c in term 1.
I20260808 07:02:01.332711 9641 raft_consensus.cc:2454] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0c5bb06ed5d47fcab2413aa14f94d4c in term 1.
I20260808 07:02:01.332896 9347 leader_election.cc:302] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0c5bb06ed5d47fcab2413aa14f94d4c, ecfe10bd50c245e5a7b99ba319e6fb2e; no voters:
I20260808 07:02:01.332988 9752 raft_consensus.cc:2781] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:01.333206 9752 raft_consensus.cc:686] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 1 LEADER]: Becoming Leader. State: Replica: e0c5bb06ed5d47fcab2413aa14f94d4c, State: Running, Role: LEADER
I20260808 07:02:01.333312 9752 consensus_queue.cc:237] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } }
I20260808 07:02:01.334160 9021 catalog_manager.cc:5495] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c reported cstate change: term changed from 0 to 1, leader changed from <none> to e0c5bb06ed5d47fcab2413aa14f94d4c (127.24.181.66). New cstate: current_term: 1 leader_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c5bb06ed5d47fcab2413aa14f94d4c" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:01.341432 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.338410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.24.181.66:43283)
0808 07:02:01.338456 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.338458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.338536 (+ 78us) server_negotiation.cc:407] Connection header received
0808 07:02:01.338629 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.338630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.338649 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.338756 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:01.339083 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.339596 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.340469 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.340717 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.340800 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.341025 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:01.341136 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:01.341244 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.341266 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.341286 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:01.343300 9641 raft_consensus.cc:1261] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 1 FOLLOWER]: Refusing update from remote peer e0c5bb06ed5d47fcab2413aa14f94d4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:01.343364 9238 raft_consensus.cc:1261] T 016147409b9c41298e061c459a02fce3 P 030c17abe22645e1af5c70ed9847d80a [term 1 FOLLOWER]: Refusing update from remote peer e0c5bb06ed5d47fcab2413aa14f94d4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:01.343649 9752 consensus_queue.cc:1035] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "030c17abe22645e1af5c70ed9847d80a" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:01.343811 9761 consensus_queue.cc:1035] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecfe10bd50c245e5a7b99ba319e6fb2e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:01.347395 9765 mvcc.cc:205] Tried to move back new op lower bound from 7316162647419555840 to 7316162647381815296. Current Snapshot: MvccSnapshot[applied={T|T < 7316162647419555840}]
I20260808 07:02:01.348894 9767 mvcc.cc:205] Tried to move back new op lower bound from 7316162647419555840 to 7316162647381815296. Current Snapshot: MvccSnapshot[applied={T|T < 7316162647419555840}]
I20260808 07:02:01.349727 9766 mvcc.cc:205] Tried to move back new op lower bound from 7316162647419555840 to 7316162647381815296. Current Snapshot: MvccSnapshot[applied={T|T < 7316162647419555840 or (T in {7316162647419555840})}]
I20260808 07:02:01.352448 9021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:01.352572 9021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:01.353365 9021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45669dbc8f6e47a0bdea4bb26600e11d: Sending DeleteTablet for 3 replicas of tablet 016147409b9c41298e061c459a02fce3
I20260808 07:02:01.353771 9215 tablet_service.cc:1514] Processing DeleteTablet for tablet 016147409b9c41298e061c459a02fce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:01 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:46942
I20260808 07:02:01.353900 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet 016147409b9c41298e061c459a02fce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:01 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:58708
I20260808 07:02:01.354010 9618 tablet_service.cc:1514] Processing DeleteTablet for tablet 016147409b9c41298e061c459a02fce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:01 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:51356
I20260808 07:02:01.354229 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 9129
I20260808 07:02:01.354282 9780 tablet_replica.cc:330] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: stopping tablet replica
I20260808 07:02:01.356889 9780 raft_consensus.cc:2229] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:01.357054 9780 raft_consensus.cc:2258] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:01.357573 9780 ts_tablet_manager.cc:1905] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:01.360038 9780 ts_tablet_manager.cc:1918] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:01.360159 9780 log.cc:1196] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/016147409b9c41298e061c459a02fce3
I20260808 07:02:01.361014 9780 ts_tablet_manager.cc:1939] T 016147409b9c41298e061c459a02fce3 P ecfe10bd50c245e5a7b99ba319e6fb2e: Deleting consensus metadata
I20260808 07:02:01.361903 9005 catalog_manager.cc:4841] TS ecfe10bd50c245e5a7b99ba319e6fb2e (127.24.181.67:41487): tablet 016147409b9c41298e061c459a02fce3 (table test-table [id=861c3ac82ba0470e9be087eaad1c7aa8]) successfully deleted
I20260808 07:02:01.363054 9778 tablet_replica.cc:330] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: stopping tablet replica
I20260808 07:02:01.363238 9778 raft_consensus.cc:2229] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:01.363446 9778 raft_consensus.cc:2258] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:01.363997 9778 ts_tablet_manager.cc:1905] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:01.366016 9778 ts_tablet_manager.cc:1918] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:01.366154 9778 log.cc:1196] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/016147409b9c41298e061c459a02fce3
I20260808 07:02:01.366539 9778 ts_tablet_manager.cc:1939] T 016147409b9c41298e061c459a02fce3 P e0c5bb06ed5d47fcab2413aa14f94d4c: Deleting consensus metadata
I20260808 07:02:01.367028 9005 catalog_manager.cc:4841] TS e0c5bb06ed5d47fcab2413aa14f94d4c (127.24.181.66:43283): tablet 016147409b9c41298e061c459a02fce3 (table test-table [id=861c3ac82ba0470e9be087eaad1c7aa8]) successfully deleted
W20260808 07:02:01.370733 9002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:36401 (error 108)
I20260808 07:02:01.371513 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 9315
I20260808 07:02:01.377683 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.377494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36401 (local address 127.0.0.1:46954)
0808 07:02:01.377571 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.377631 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 07:02:01.377843 9002 catalog_manager.cc:4584] TS 030c17abe22645e1af5c70ed9847d80a (127.24.181.65:36401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 016147409b9c41298e061c459a02fce3: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36401: connect: Connection refused (error 111)
I20260808 07:02:01.387158 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 9520
I20260808 07:02:01.409019 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 8975
I20260808 07:02:01.420768 25301 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:02:01.764950 25301 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:02:01.765061 25301 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:02:02.119279 25301 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:02:02.119 UTC [8403] LOG: received smart shutdown request
2026-08-08 07:02:02.123 UTC [8403] LOG: background worker "logical replication launcher" (PID 8419) exited with exit code 1
2026-08-08 07:02:02.124 UTC [8411] LOG: shutting down
2026-08-08 07:02:02.298 UTC [8403] LOG: database system is shut down
2026-08-08T07:02:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53670 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:02 dist-test-slave-nfb5 krb5kdc[9796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:02 dist-test-slave-nfb5 krb5kdc[9796](info): set up 2 sockets
Aug 08 07:02:02 dist-test-slave-nfb5 krb5kdc[9796](info): commencing operation
krb5kdc: starting...
W20260808 07:02:03.680835 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:02:03 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangeradmin_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangeradmin_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangerlookup_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangerlookup_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
I20260808 07:02:03.850069 25301 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNKFpcq/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/postgres -l logfile start
2026-08-08 07:02:06.337 UTC [9836] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:02:06.337 UTC [9836] LOG: listening on IPv4 address "127.24.181.84", port 35033
2026-08-08 07:02:06.341 UTC [9836] LOG: listening on Unix socket "/tmp/.s.PGSQL.35033"
2026-08-08 07:02:06.377 UTC [9839] LOG: database system was shut down at 2026-08-08 07:02:04 UTC
2026-08-08 07:02:06.381 UTC [9836] LOG: database system is ready to accept connections
I20260808 07:02:07.590976 25301 mini_postgres.cc:96] Postgres bound to 35033
2026-08-08 07:02:07.597 UTC [9852] FATAL: database "slave" does not exist
127.24.181.84:35033 - accepting connections
I20260808 07:02:07.598456 25301 mini_ranger.cc:161] Starting Ranger...
I20260808 07:02:07.614037 25301 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:02:08.076309 25301 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:02:08.076413 25301 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin
2026-08-08 07:02:08,371 [I] DB FLAVOR :POSTGRES
2026-08-08 07:02:08,372 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:02:08,372 [I] Checking connection..
2026-08-08 07:02:08,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:02:08,646 [I] Checking connection passed.
2026-08-08 07:02:08,646 [I] --------- Verifying version history table ---------
2026-08-08 07:02:08,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:08,896 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:02:08,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:09,155 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:02:09,155 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:02:09,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:02:09.416 UTC [9946] WARNING: there is no transaction in progress
2026-08-08 07:02:09,424 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:02:09,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:09,669 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:02:09,670 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:02:09,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:02:09,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:02:10,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nfb5.c.gcp-upstream.internal','N') ;"
2026-08-08 07:02:10,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:02:10,725 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:02:10,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:02:10,983 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:02:10,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:02:11,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:02:11,490 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:02:11,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:02:12.467 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.484 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.501 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.516 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.532 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.552 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.616 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.623 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.638 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.648 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.659 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.671 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.682 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.688 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.694 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:12.701 UTC [10109] WARNING: there is no transaction in progress
2026-08-08 07:02:13,330 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:02:13,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:02:13,592 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:02:13,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:02:13,846 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:02:13,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:02:14,110 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:02:14,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:02:14,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:02:14,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nfb5.c.gcp-upstream.internal';"
2026-08-08 07:02:14,915 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:02:14,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:02:15,183 [I] DB_PATCHES have already been applied
I20260808 07:02:15.189320 25301 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNKFpcq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:02:15.189393 25301 mini_ranger.cc:193] Using host: 127.24.181.84
I20260808 07:02:15.193053 25301 mini_ranger.cc:227] Ranger admin URL: http://127.24.181.84:41945
Aug 08, 2026 7:02:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:02:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:02:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:02:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:02:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:02:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:02:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:02:16.602669 25301 mini_ranger.cc:160] Time spent starting Ranger: real 9.004s user 0.000s sys 0.010s
Aug 08, 2026 7:02:36 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:02:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 7:02:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:02:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:02:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:02:38.608953 25301 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangerkms_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/rangerkms_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.181.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.181.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.181.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/HTTP_127.24.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:02:38 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172558, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:02:38.712559 25301 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:02:38.723619 25301 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:02:39.564618 25301 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:02:39.564716 25301 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms
I20260808 07:02:39.564739 25301 mini_ranger_kms.cc:227] Using postgres at 127.24.181.84:35033
2026-08-08 07:02:39,744 [I] DB FLAVOR :POSTGRES
2026-08-08 07:02:39,744 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:02:39,745 [I] Checking connection
2026-08-08 07:02:39,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:02:39,994 [I] connection success
2026-08-08 07:02:39,994 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:02:39,994 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:02:39,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:02:40,249 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:02:40,249 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:02:40,250 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:02:40,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.181.84:35033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:02:40,537 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:02:41.016217 25301 mini_ranger_kms.cc:310] Created kms service
I20260808 07:02:41.213587 25301 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:02:41.259246 25301 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:02:41 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172558, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:41.693638 25301 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:02:41.693864 25301 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNKFpcq/build/release/bin/postgresql.jar:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNKFpcq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:02:41.693889 25301 mini_ranger_kms.cc:240] Using host: 127.24.181.84
I20260808 07:02:41.698033 25301 mini_ranger_kms.cc:276] Ranger KMS PID: 10365
I20260808 07:02:41.698130 25301 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.181.84:36575
Aug 08, 2026 7:02:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:02:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:02:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:02:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:02:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:02:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 07:02:44 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172564, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:44.378822 25301 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.666s user 0.003s sys 0.159s
I20260808 07:02:44.378938 25301 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:02:44.378989 25301 mini_ranger_kms.cc:401] 127.24.181.84:36575/kms/v1/keys
Aug 08 07:02:44 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172564, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
Aug 08 07:02:44 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172564, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.181.84@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
Aug 08 07:02:47 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:02:47.329097 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33813
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:33813
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:47.411347 10434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:47.411542 10434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:47.411564 10434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:47.412679 10434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:47.412722 10434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:47.412736 10434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:47.412748 10434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:47.412760 10434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:47.414395 10434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.181.84:36575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:33813
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:47.414638 10434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:47.414896 10434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:02:47.421128 10434 server_base.cc:1034] running on GCE node
W20260808 07:02:47.421100 10443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:47.421267 10448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:47.421089 10442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:47.421650 10434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:47.421967 10434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:47.423115 10434 hybrid_clock.cc:648] HybridClock initialized: now 1786172567423083 us; error 52 us; skew 500 ppm
Aug 08 07:02:47 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172567, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:47.429970 10434 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:47.430508 10434 webserver.cc:460] Webserver started at http://127.24.181.126:35215/ using document root <none> and password file <none>
I20260808 07:02:47.430744 10434 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:47.430784 10434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:47.430964 10434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:47 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172567, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:47.487831 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "dda19c98fa8e41c19707f7191a7d226a"
format_stamp: "Formatted at 2026-08-08 07:02:47 on dist-test-slave-nfb5"
server_key: "a900994688147bb8ee385b4061d21903"
server_key_iv: "f6b725035be94617fac0d7e6f57d481f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:47.488313 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "dda19c98fa8e41c19707f7191a7d226a"
format_stamp: "Formatted at 2026-08-08 07:02:47 on dist-test-slave-nfb5"
server_key: "a900994688147bb8ee385b4061d21903"
server_key_iv: "f6b725035be94617fac0d7e6f57d481f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:47.490336 10434 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:47.505842 10459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:47.506119 10434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:47.506260 10434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "dda19c98fa8e41c19707f7191a7d226a"
format_stamp: "Formatted at 2026-08-08 07:02:47 on dist-test-slave-nfb5"
server_key: "a900994688147bb8ee385b4061d21903"
server_key_iv: "f6b725035be94617fac0d7e6f57d481f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:47.506378 10434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:47.534024 10434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:47.536145 10434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:47.536314 10434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:47.540844 10434 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:33813
I20260808 07:02:47.540890 10537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:33813 every 8 connection(s)
I20260808 07:02:47.541263 10434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:47.541987 10538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:47.544483 10538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Bootstrap starting.
I20260808 07:02:47.545095 10538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:47.545400 10538 log.cc:824] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:47.545679 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 10434
I20260808 07:02:47.545809 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:02:47.546046 25301 external_mini_cluster.cc:1428] Setting key 832ab36ca23e5192c412716a4bf83329
I20260808 07:02:47.546233 10538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: No bootstrap required, opened a new log
I20260808 07:02:47.548718 10538 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:47.548882 10538 raft_consensus.cc:374] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:47.548949 10538 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda19c98fa8e41c19707f7191a7d226a, State: Initialized, Role: FOLLOWER
I20260808 07:02:47.549108 10538 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:47.549209 10538 raft_consensus.cc:388] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:47.549268 10538 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:47.549333 10538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:47.549897 10538 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:47.550024 10538 leader_election.cc:302] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dda19c98fa8e41c19707f7191a7d226a; no voters:
I20260808 07:02:47.550196 10538 leader_election.cc:288] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:47.550271 10545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:47.550730 10538 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:47.550735 10545 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 LEADER]: Becoming Leader. State: Replica: dda19c98fa8e41c19707f7191a7d226a, State: Running, Role: LEADER
I20260808 07:02:47.550907 10545 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:47.551311 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda19c98fa8e41c19707f7191a7d226a. Latest consensus state: current_term: 1 leader_uuid: "dda19c98fa8e41c19707f7191a7d226a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } } }
I20260808 07:02:47.551394 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:47.551322 10548 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dda19c98fa8e41c19707f7191a7d226a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } } }
I20260808 07:02:47.552621 10548 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:47.552887 10538 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 08 07:02:47 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:47.556787 10550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:47.558153 10550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:47.562966 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:47.546585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.24.181.126:33813)
0808 07:02:47.546795 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:47.546805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:47.546841 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:02:47.548737 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:47.548747 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:47.548828 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:47.548991 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:47.550002 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:47.551425 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:47.552501 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:47.553163 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:47.553272 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:47.557075 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:47.557106 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:47.557106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:47.557135 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:47.560218 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:47.560674 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:47.560678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:47.560678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:47.560792 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:47.561037 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:47.561038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:47.561038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:47.561469 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:47.561630 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:47.562249 (+ 619us) server_negotiation.cc:299] Negotiation successful
0808 07:02:47.562590 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260808 07:02:47.569876 10550 catalog_manager.cc:1347] Generated new cluster ID: 4748ae8b3c8b4533acd89df4b62cf821
I20260808 07:02:47.569947 10550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:47.590137 10550 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:47.590852 10550 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:47.598354 10550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Generated new TSK 0
I20260808 07:02:47.598537 10550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:02:48.325702 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:48.428427 10600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:48.428727 10600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:48.428762 10600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:48.430573 10600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:48.430639 10600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:48.430678 10600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:02:48.432964 10600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:48.433383 10600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:48.433657 10600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:48.439736 10609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:48.439894 10612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:48.439765 10610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:48.440603 10600 server_base.cc:1034] running on GCE node
I20260808 07:02:48.440841 10600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:48.441090 10600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:48.442252 10600 hybrid_clock.cc:648] HybridClock initialized: now 1786172568442229 us; error 43 us; skew 500 ppm
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:48.444859 10600 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:02:48.445365 10600 webserver.cc:460] Webserver started at http://127.24.181.65:37695/ using document root <none> and password file <none>
I20260808 07:02:48.445636 10600 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:48.445684 10600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:48.445832 10600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:48.464401 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "5304bea8f60144b99e3904f029ba08e0"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "86ebaceb6899a830292458a5215f8b45"
server_key_iv: "bbb033d421e832ceb1aaae0da63b6a03"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.464952 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "5304bea8f60144b99e3904f029ba08e0"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "86ebaceb6899a830292458a5215f8b45"
server_key_iv: "bbb033d421e832ceb1aaae0da63b6a03"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.472882 10600 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260808 07:02:48.492377 10627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.492559 10600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:48.492650 10600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "5304bea8f60144b99e3904f029ba08e0"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "86ebaceb6899a830292458a5215f8b45"
server_key_iv: "bbb033d421e832ceb1aaae0da63b6a03"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.492762 10600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:48.514021 10600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:48.516758 10600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:48.516964 10600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:48.517269 10600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:48.517699 10600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:48.517791 10600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.517891 10600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:48.517966 10600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.527217 10600 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36861
I20260808 07:02:48.527716 10600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:48.528961 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36861 every 8 connection(s)
I20260808 07:02:48.531421 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 10600
I20260808 07:02:48.531520 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:48.531720 25301 external_mini_cluster.cc:1428] Setting key acc186c142b3821a030e728f0b75a16f
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:48.543675 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:48.528672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.65:38531)
0808 07:02:48.531088 (+ 2416us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:48.531143 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:48.532215 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:48.532632 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:48.532637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:48.533025 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:48.533390 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:48.533397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.534544 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.534545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:48.536131 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:48.536134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.536688 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.536689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:48.536861 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:48.537560 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:48.537571 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:48.539522 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:48.541758 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:48.541766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:48.541768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:48.542180 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:48.542439 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:48.542440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:48.542440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:48.542578 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:48.542981 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:48.542989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:48.543160 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 07:02:48.543268 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:02:48.543439 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2331,"thread_start_us":269,"threads_started":1}
I20260808 07:02:48.544255 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:48.528879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38531 (local address 127.24.181.126:33813)
0808 07:02:48.531073 (+ 2194us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:48.531076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:48.532011 (+ 935us) server_negotiation.cc:407] Connection header received
0808 07:02:48.532389 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:48.532391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:48.532406 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:48.532489 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:48.533532 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.534414 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.536235 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.536543 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.536630 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:48.539634 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:48.539635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:48.539636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:48.539653 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:48.541662 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:48.542282 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:48.542284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:48.542284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:48.542354 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:48.542684 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:48.542686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:48.542686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:48.542862 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:48.542968 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:48.544071 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0808 07:02:48.544161 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":91,"threads_started":1}
I20260808 07:02:48.544942 10790 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:48.545070 10790 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:48.545270 10790 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:48.545827 10486 ts_manager.cc:194] Registered new tserver with Master: 5304bea8f60144b99e3904f029ba08e0 (127.24.181.65:36861)
I20260808 07:02:48.546725 10486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:38531
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:02:48.580524 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 07:02:48.769385 10796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:48.769694 10796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:48.769743 10796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:48.771842 10796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:48.771924 10796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:48.771963 10796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:02:48.774523 10796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:48.774821 10796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:48.775122 10796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:48.781580 10806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:48.781577 10809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:48.781874 10796 server_base.cc:1034] running on GCE node
W20260808 07:02:48.782076 10805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:48.782302 10796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:48.782553 10796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:48.783710 10796 hybrid_clock.cc:648] HybridClock initialized: now 1786172568783699 us; error 39 us; skew 500 ppm
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:48.787456 10796 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:02:48.787964 10796 webserver.cc:460] Webserver started at http://127.24.181.66:38111/ using document root <none> and password file <none>
I20260808 07:02:48.788246 10796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:48.788321 10796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:48.788470 10796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:48.813292 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "386d5ae613714808acde3fb54196c344"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "c4d5bf6875813742b6bc304226f01901"
server_key_iv: "7d44d65302f7a5ad803b9f6f4a0e5026"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.813827 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "386d5ae613714808acde3fb54196c344"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "c4d5bf6875813742b6bc304226f01901"
server_key_iv: "7d44d65302f7a5ad803b9f6f4a0e5026"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.816310 10796 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:02:48.841373 10818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.842141 10796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:02:48.842273 10796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "386d5ae613714808acde3fb54196c344"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "c4d5bf6875813742b6bc304226f01901"
server_key_iv: "7d44d65302f7a5ad803b9f6f4a0e5026"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:48.842398 10796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:48.891501 10796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:48.894138 10796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:48.894316 10796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:48.894584 10796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:48.894994 10796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:48.895063 10796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.895100 10796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:48.895123 10796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:48.910259 10796 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:41941
I20260808 07:02:48.910843 10796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:48.912089 10984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:41941 every 8 connection(s)
I20260808 07:02:48.914731 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 10796
I20260808 07:02:48.914872 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:48.915125 25301 external_mini_cluster.cc:1428] Setting key eeff95425fab1d689c961a680cda332b
Aug 08 07:02:48 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:48.926218 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:48.913498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60305 (local address 127.24.181.126:33813)
0808 07:02:48.913586 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:48.913589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:48.913915 (+ 326us) server_negotiation.cc:407] Connection header received
0808 07:02:48.915043 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:48.915044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:48.915064 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:48.915101 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:48.916321 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.917151 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.918972 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.919371 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.919422 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:48.922828 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:48.922829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:48.922830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:48.922839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:48.924631 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:48.925132 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:48.925133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:48.925133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:48.925194 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:48.925468 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:48.925469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:48.925469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:48.925605 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:48.925662 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:48.926026 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 07:02:48.926119 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:02:48.927079 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:48.913504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.66:60305)
0808 07:02:48.913754 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:48.913800 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:48.914878 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:48.915305 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:48.915309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:48.915766 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:48.916197 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:48.916204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.917274 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.917275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:48.918827 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:48.918831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:48.919584 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:48.919585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:48.919785 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:48.920555 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:48.920565 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:48.922739 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:48.924745 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:48.924753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:48.924755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:48.925040 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:48.925263 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:48.925264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:48.925264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:48.925390 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:48.925711 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:48.925718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:48.925877 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 07:02:48.926566 (+ 689us) client_negotiation.cc:240] Negotiation successful
0808 07:02:48.926741 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
I20260808 07:02:48.927860 10985 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:48.927959 10985 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:48.928159 10985 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:48.928573 10486 ts_manager.cc:194] Registered new tserver with Master: 386d5ae613714808acde3fb54196c344 (127.24.181.66:41941)
I20260808 07:02:48.929018 10486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:60305
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:02:48.956751 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:49.128485 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:49.128885 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:49.128998 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:49.130856 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:49.130977 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:49.131042 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:02:49.132994 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:49.133361 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:49.133631 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:49.144083 10996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:49.144219 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.144510 10991 server_base.cc:1034] running on GCE node
W20260808 07:02:49.144760 11001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.145040 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:49.145342 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:49.146524 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786172569146508 us; error 38 us; skew 500 ppm
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:49.150234 10991 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:02:49.150672 10991 webserver.cc:460] Webserver started at http://127.24.181.67:36863/ using document root <none> and password file <none>
I20260808 07:02:49.150978 10991 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:49.151057 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:49.151206 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:49.165464 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "9b205656263147dbb5198925aaeeca07"
format_stamp: "Formatted at 2026-08-08 07:02:49 on dist-test-slave-nfb5"
server_key: "8cdde8c787bf3c7c1811031700cae502"
server_key_iv: "d487acc67e43185314e6e8d797d2002f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.171149 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "9b205656263147dbb5198925aaeeca07"
format_stamp: "Formatted at 2026-08-08 07:02:49 on dist-test-slave-nfb5"
server_key: "8cdde8c787bf3c7c1811031700cae502"
server_key_iv: "d487acc67e43185314e6e8d797d2002f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.173557 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:02:49.186980 11016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.187783 10991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:02:49.187897 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "9b205656263147dbb5198925aaeeca07"
format_stamp: "Formatted at 2026-08-08 07:02:49 on dist-test-slave-nfb5"
server_key: "8cdde8c787bf3c7c1811031700cae502"
server_key_iv: "d487acc67e43185314e6e8d797d2002f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.187987 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:49.207149 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:49.209604 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:49.209793 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:49.210110 10991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:49.210541 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:49.210588 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.210620 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:49.210638 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.221407 10991 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:36517
I20260808 07:02:49.221918 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:49.225649 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 10991
I20260808 07:02:49.225764 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:49.225957 25301 external_mini_cluster.cc:1428] Setting key a6f7c2edad951656323b293d2ae0cf28
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:49.223591 11170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:36517 every 8 connection(s)
I20260808 07:02:49.236414 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.223157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:44283 (local address 127.24.181.126:33813)
0808 07:02:49.223207 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:49.223209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:49.224079 (+ 870us) server_negotiation.cc:407] Connection header received
0808 07:02:49.224839 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:49.224841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:49.224854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:49.224889 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:49.227316 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.228105 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.229615 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.229923 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.229971 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.232699 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:49.232700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:49.232701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:49.232708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:49.234584 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.235245 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.235258 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.235258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.235324 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.235626 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.235627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.235628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.235770 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:49.235830 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:49.236205 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 07:02:49.236318 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:02:49.237901 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.222900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.67:44283)
0808 07:02:49.223937 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:49.223985 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:49.224704 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:49.225087 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:49.225092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:49.225363 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:49.227195 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.227202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.228224 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.228225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.229506 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.229510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.230082 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.230082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.230225 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.230953 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:49.230963 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:49.232580 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:49.234806 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.234811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.234814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.235145 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.235401 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.235402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.235402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.235539 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.235869 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:49.235876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:49.236036 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:02:49.236827 (+ 791us) client_negotiation.cc:240] Negotiation successful
0808 07:02:49.237617 (+ 790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":83,"threads_started":1}
I20260808 07:02:49.238570 11171 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:49.238663 11171 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.238843 11171 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:49.239322 10486 ts_manager.cc:194] Registered new tserver with Master: 9b205656263147dbb5198925aaeeca07 (127.24.181.67:36517)
I20260808 07:02:49.239821 10486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:44283
I20260808 07:02:49.248312 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:49.248997 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 10600
I20260808 07:02:49.269272 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 10796
I20260808 07:02:49.281200 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 10991
I20260808 07:02:49.322187 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 10434
I20260808 07:02:49.339689 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33813
--webserver_interface=127.24.181.126
--webserver_port=35215
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:33813
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:49.435282 11174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:49.435494 11174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:49.435516 11174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:49.436656 11174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:49.436698 11174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:49.436712 11174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:49.436724 11174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:49.436738 11174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:49.438367 11174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:33813
--ranger_config_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:49.438633 11174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:49.438868 11174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:49.443907 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:49.443917 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.444161 11174 server_base.cc:1034] running on GCE node
W20260808 07:02:49.444231 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.444476 11174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:49.444761 11174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:49.445919 11174 hybrid_clock.cc:648] HybridClock initialized: now 1786172569445907 us; error 47 us; skew 500 ppm
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:49.448514 11174 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:49.448943 11174 webserver.cc:460] Webserver started at http://127.24.181.126:35215/ using document root <none> and password file <none>
I20260808 07:02:49.449170 11174 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:49.449244 11174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:49.518096 11174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:49.518868 11197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.519052 11174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:49.519163 11174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "dda19c98fa8e41c19707f7191a7d226a"
format_stamp: "Formatted at 2026-08-08 07:02:47 on dist-test-slave-nfb5"
server_key: "a900994688147bb8ee385b4061d21903"
server_key_iv: "f6b725035be94617fac0d7e6f57d481f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.519532 11174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:49.528666 11174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:49.530185 11174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:49.530354 11174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:49.534652 11174 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:33813
I20260808 07:02:49.534704 11264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:33813 every 8 connection(s)
I20260808 07:02:49.535086 11174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:49.536273 11265 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:02:49.536672 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 11174
I20260808 07:02:49.536960 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:36861
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:33813
--webserver_port=37695
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:49.536985 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Bootstrap starting.
I20260808 07:02:49.545596 11265 log.cc:824] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:49.547390 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:49.547670 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Bootstrap complete.
I20260808 07:02:49.550093 11265 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:49.550350 11265 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda19c98fa8e41c19707f7191a7d226a, State: Initialized, Role: FOLLOWER
I20260808 07:02:49.550524 11265 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:49.550598 11265 raft_consensus.cc:388] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:49.550652 11265 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:49.550699 11265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:02:49.551756 11265 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:49.551880 11265 leader_election.cc:302] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dda19c98fa8e41c19707f7191a7d226a; no voters:
I20260808 07:02:49.552058 11265 leader_election.cc:288] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:02:49.552132 11271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:02:49.552274 11271 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [term 2 LEADER]: Becoming Leader. State: Replica: dda19c98fa8e41c19707f7191a7d226a, State: Running, Role: LEADER
I20260808 07:02:49.552361 11265 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:49.552371 11271 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } }
I20260808 07:02:49.552649 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dda19c98fa8e41c19707f7191a7d226a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } } }
I20260808 07:02:49.552647 11271 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda19c98fa8e41c19707f7191a7d226a. Latest consensus state: current_term: 2 leader_uuid: "dda19c98fa8e41c19707f7191a7d226a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda19c98fa8e41c19707f7191a7d226a" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 33813 } } }
I20260808 07:02:49.552894 11271 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:49.553103 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:49.553242 11281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:49.553705 11281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:49.553840 11281 catalog_manager.cc:1259] Loaded cluster ID: 4748ae8b3c8b4533acd89df4b62cf821
I20260808 07:02:49.553877 11281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:49.554086 11265 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:02:49.558305 11281 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:49.558903 11281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Loaded TSK: 0
I20260808 07:02:49.559211 11281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:02:49.653591 11268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:49.653872 11268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:49.653911 11268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:49.655661 11268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:49.655725 11268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:49.655766 11268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:02:49.657966 11268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:36861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=37695
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:49.658216 11268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:49.658464 11268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:49.668066 11316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.668490 11268 server_base.cc:1034] running on GCE node
W20260808 07:02:49.668606 11313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:49.668781 11318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.669059 11268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:49.669384 11268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:49.670578 11268 hybrid_clock.cc:648] HybridClock initialized: now 1786172569670548 us; error 45 us; skew 500 ppm
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:49.674039 11268 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:02:49.674582 11268 webserver.cc:460] Webserver started at http://127.24.181.65:37695/ using document root <none> and password file <none>
I20260808 07:02:49.674880 11268 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:49.675019 11268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:49.704752 11268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:49.706036 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.706316 11268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:49.706429 11268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "5304bea8f60144b99e3904f029ba08e0"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "86ebaceb6899a830292458a5215f8b45"
server_key_iv: "bbb033d421e832ceb1aaae0da63b6a03"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.706837 11268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:49.725199 11268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:49.727269 11268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:49.727435 11268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:49.727736 11268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:49.728068 11268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:49.728138 11268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.728173 11268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:49.728199 11268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.735739 11268 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:36861
I20260808 07:02:49.735872 11486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:36861 every 8 connection(s)
I20260808 07:02:49.736172 11268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:49.737980 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 11268
I20260808 07:02:49.738324 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:41941
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:33813
--webserver_port=38111
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:49.750695 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.736860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.65:37403)
0808 07:02:49.737065 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:49.737116 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:49.737827 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:49.738690 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:49.738695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:49.738978 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:49.739279 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.739287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.741028 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.741029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.742276 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.742279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.743445 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.743447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.743574 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.744281 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:49.744291 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:49.746002 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:49.748922 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.748930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.748932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.749164 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.749453 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.749454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.749455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.749574 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.749992 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:49.750000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:49.750161 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 07:02:49.750279 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:02:49.750461 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260808 07:02:49.751353 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.737038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:37403 (local address 127.24.181.126:33813)
0808 07:02:49.737268 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:49.737282 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:49.737312 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:02:49.738140 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:49.738148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:49.738216 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:49.738387 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:49.739434 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.740894 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.742833 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.743305 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.743453 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.746134 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:49.746139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:49.746139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:49.746151 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:49.748811 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.749267 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.749271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.749271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.749375 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.749652 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.749653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.749653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.749886 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:49.749973 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:49.751028 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0808 07:02:49.751157 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":127,"threads_started":1}
I20260808 07:02:49.752015 11487 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:49.752115 11487 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.752301 11487 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:49.753237 11487 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:33813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:02:49.753315 11487 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.753381 11487 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:49.753715 11487 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.753799 11487 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:49.754150 11487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:02:49.867153 11492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:49.867440 11492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:49.867489 11492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:49.869180 11492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:49.869246 11492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:49.869285 11492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:02:49.871543 11492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:41941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=38111
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:49.871905 11492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:49.872229 11492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:49.878038 11497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:49.880863 11498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:49.881376 11500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:49.881721 11492 server_base.cc:1034] running on GCE node
I20260808 07:02:49.881948 11492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:49.882231 11492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:49.883400 11492 hybrid_clock.cc:648] HybridClock initialized: now 1786172569883144 us; error 271 us; skew 500 ppm
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:49.886699 11492 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:02:49.887226 11492 webserver.cc:460] Webserver started at http://127.24.181.66:38111/ using document root <none> and password file <none>
I20260808 07:02:49.887476 11492 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:49.887539 11492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:49.903010 11492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:49.905308 11512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.905520 11492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:49.905618 11492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "386d5ae613714808acde3fb54196c344"
format_stamp: "Formatted at 2026-08-08 07:02:48 on dist-test-slave-nfb5"
server_key: "c4d5bf6875813742b6bc304226f01901"
server_key_iv: "7d44d65302f7a5ad803b9f6f4a0e5026"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:49.906065 11492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:49.929589 11492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:49.931844 11492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:49.932026 11492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:49.932327 11492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:49.932723 11492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:49.932780 11492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.932816 11492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:49.932843 11492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:49.940752 11492 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:41941
I20260808 07:02:49.941241 11492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:49.941886 11664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:41941 every 8 connection(s)
I20260808 07:02:49.947252 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 11492
I20260808 07:02:49.947556 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.181.84:36575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:36517
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:33813
--webserver_port=36863
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:49 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:49.951665 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.942090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37077 (local address 127.24.181.126:33813)
0808 07:02:49.942141 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:49.942143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:49.942493 (+ 350us) server_negotiation.cc:407] Connection header received
0808 07:02:49.943206 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:49.943208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:49.943223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:49.943283 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:49.944037 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.944609 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.945608 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.945835 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.945921 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.948468 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:49.948470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:49.948470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:49.948479 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:49.950243 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.950652 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.950654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.950654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.950715 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:49.951005 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:49.951006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:49.951007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:49.951124 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:49.951201 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:49.951478 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:02:49.951573 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:02:49.951853 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:49.942103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.66:37077)
0808 07:02:49.942326 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:49.942375 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:49.943089 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:49.943385 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:49.943391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:49.943629 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:49.943927 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.943934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.944707 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.944708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.945511 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:49.945513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:49.945958 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:49.945959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:49.946081 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:49.946622 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:49.946631 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:49.948290 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:49.950338 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.950346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.950349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.950575 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.950791 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:49.950792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:49.950792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:49.950946 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:49.951192 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:49.951200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:49.951345 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 07:02:49.951444 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:02:49.951595 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260808 07:02:49.952535 11665 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:49.952656 11665 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.952849 11665 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:49.953573 11665 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:33813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:02:49.953655 11665 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.953728 11665 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:49.954020 11665 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:49.954095 11665 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:49.954367 11665 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:02:50.071626 11669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:50.071935 11669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:50.071982 11669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:50.073743 11669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:50.073807 11669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:50.073849 11669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:02:50.076100 11669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.181.84:36575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:36517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=36863
--tserver_master_addrs=127.24.181.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:50.076368 11669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:50.076632 11669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:50.084967 11682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:50.085736 11669 server_base.cc:1034] running on GCE node
W20260808 07:02:50.086555 11684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:50.086756 11681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:50.087095 11669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:50.087394 11669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:50.090715 11669 hybrid_clock.cc:648] HybridClock initialized: now 1786172570090663 us; error 64 us; skew 500 ppm
Aug 08 07:02:50 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:50.096675 11669 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:02:50.097134 11669 webserver.cc:460] Webserver started at http://127.24.181.67:36863/ using document root <none> and password file <none>
I20260808 07:02:50.097404 11669 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:50.097498 11669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:02:50 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:50.114979 11669 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 07:02:50.117153 11697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:50.117370 11669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:50.117463 11669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "9b205656263147dbb5198925aaeeca07"
format_stamp: "Formatted at 2026-08-08 07:02:49 on dist-test-slave-nfb5"
server_key: "8cdde8c787bf3c7c1811031700cae502"
server_key_iv: "d487acc67e43185314e6e8d797d2002f"
server_key_version: "kuduclusterkey@0"
I20260808 07:02:50.117842 11669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:50.135839 11669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:50.138196 11669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:50.138365 11669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:50.138602 11669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:50.138967 11669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:50.139019 11669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:50.139055 11669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:50.139073 11669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:50.146907 11669 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:36517
I20260808 07:02:50.147023 11872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:36517 every 8 connection(s)
I20260808 07:02:50.147358 11669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:02:50 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:50.155745 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 11669
I20260808 07:02:50.157925 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:50.148128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:45117 (local address 127.24.181.126:33813)
0808 07:02:50.148187 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:50.148189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:50.148416 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:02:50.148912 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:50.148914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:50.148927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:50.149005 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:50.149516 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.150237 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.151303 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.151589 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.151663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:50.154321 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:50.154323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:50.154323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:50.154332 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:50.156033 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:50.156806 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:50.156807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:50.156808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:50.156871 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:50.157181 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:50.157182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:50.157182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:50.157313 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:50.157396 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:50.157738 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 07:02:50.157834 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:02:50.158653 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:50.148098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:33813 (local address 127.24.181.67:45117)
0808 07:02:50.148277 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:50.148319 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:50.148786 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:50.149060 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:50.149063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:50.149231 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:50.149423 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:50.149427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.150331 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.150332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:50.151209 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:50.151212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.151740 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.151742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:50.151903 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:50.152573 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:50.152582 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:50.154191 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:50.156368 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:50.156377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:50.156380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:50.156689 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:50.156945 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:50.156946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:50.156947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:50.157083 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:50.157405 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:50.157413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:50.157574 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 07:02:50.157706 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 07:02:50.158419 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260808 07:02:50.159214 11873 heartbeater.cc:347] Connected to a master server at 127.24.181.126:33813
I20260808 07:02:50.159312 11873 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:50.159518 11873 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:50.160516 11873 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:33813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:02:50.160581 11873 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:50.160626 11873 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:50.163122 11873 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:50.163210 11873 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
W20260808 07:02:50.163834 11873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:02:50.169391 11879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:50.156307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.24.181.126:33813)
0808 07:02:50.156433 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:50.156435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:50.156442 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:02:50.156461 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:50.156462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:50.156473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:50.156536 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:50.157144 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.157762 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.159064 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:50.159394 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:50.159463 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:50.163079 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:50.163081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:50.163081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:50.163286 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:50.165075 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:50.168419 (+ 3344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:50.168421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:50.168421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:50.168493 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:50.168740 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:50.168741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:50.168742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:50.168917 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:50.169107 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:50.169184 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:02:50.169282 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
Aug 08 07:02:50 dist-test-slave-nfb5 krb5kdc[9796](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:50.754334 11487 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:50.754527 11487 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:50.754993 11219 ts_manager.cc:194] Registered new tserver with Master: 5304bea8f60144b99e3904f029ba08e0 (127.24.181.65:36861)
I20260808 07:02:50.756060 11219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:37403
I20260808 07:02:50.954568 11665 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:50.954741 11665 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:50.955087 11219 ts_manager.cc:194] Registered new tserver with Master: 386d5ae613714808acde3fb54196c344 (127.24.181.66:41941)
I20260808 07:02:50.955670 11219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:37077
Aug 08 07:02:51 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:51.164036 11873 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:51.164206 11873 heartbeater.cc:511] Master 127.24.181.126:33813 requested a full tablet report, sending...
I20260808 07:02:51.164613 11219 ts_manager.cc:194] Registered new tserver with Master: 9b205656263147dbb5198925aaeeca07 (127.24.181.67:36517)
I20260808 07:02:51.165185 11219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:45117
I20260808 07:02:51.166733 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:51.176934 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.167840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46396 (local address 127.24.181.126:33813)
0808 07:02:51.168161 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.168163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.168262 (+ 99us) server_negotiation.cc:407] Connection header received
0808 07:02:51.168356 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.168357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.168372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.168421 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:51.169379 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.170199 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.171435 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.171747 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.171794 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.173568 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:51.173569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:51.173569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:51.173578 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:51.175605 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:51.176065 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:51.176066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:51.176067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:51.176135 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:51.176349 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:51.176350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:51.176350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:51.176470 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:51.176539 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.176726 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.176810 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":72,"threads_started":1}
I20260808 07:02:51.179706 11219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:51.182089 11219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:51.194151 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.188913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:41941 (local address 127.0.0.1:42916)
0808 07:02:51.189553 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.189561 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:51.189593 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:51.190034 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:51.190037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:51.190039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:51.190107 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.190109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.191430 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.191431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.192792 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.192795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.193850 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.193851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.193933 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.193946 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:02:51.194011 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:02:51.194060 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":102,"threads_started":1}
I20260808 07:02:51.194499 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.189159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.24.181.66:41941)
0808 07:02:51.189801 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.189806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.189816 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:51.189851 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.189853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.189869 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.189935 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:51.190182 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.191336 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.192898 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.193662 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.193755 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.193813 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.194374 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.194417 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":51,"threads_started":1}
I20260808 07:02:51.195194 11563 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 (DEFAULT_TABLE table=test-table [id=0c4fed1b4e124c308c623ecbe717ada8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:51.195587 11563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3f98f1c7ee47cf9007cc8496936815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:51.203171 11920 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Bootstrap starting.
I20260808 07:02:51.203955 11920 tablet_bootstrap.cc:654] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:51.204280 11920 log.cc:824] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:51.205235 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.188893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36861 (local address 127.0.0.1:46458)
0808 07:02:51.189345 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.189386 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:51.189454 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:51.199679 (+ 10225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:51.199680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:51.199683 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:51.199754 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.199756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.201099 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.201100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.202235 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.202237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.205001 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.205002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.205077 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.205082 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:51.205113 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:02:51.205155 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":77,"threads_started":1}
I20260808 07:02:51.205567 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.189431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.24.181.65:36861)
0808 07:02:51.199500 (+ 10069us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.199505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.199517 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:02:51.199551 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.199552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.199570 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.199825 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:51.199914 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.200994 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.202341 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.204908 (+ 2567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.205351 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.205423 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.205449 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.205490 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10029,"thread_start_us":69,"threads_started":1}
I20260808 07:02:51.206249 11397 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 (DEFAULT_TABLE table=test-table [id=0c4fed1b4e124c308c623ecbe717ada8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:51.206687 11397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3f98f1c7ee47cf9007cc8496936815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:51.208142 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.189126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:36517 (local address 127.0.0.1:50068)
0808 07:02:51.189656 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.189663 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:51.189693 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:51.203319 (+ 13626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:51.203320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:51.203322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:51.203380 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.203382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.205699 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.205709 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.206788 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.206791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.207912 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.207913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.207989 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.207994 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:51.208025 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:02:51.208062 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":77,"threads_started":1}
I20260808 07:02:51.208520 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.189595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.24.181.67:36517)
0808 07:02:51.203112 (+ 13517us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.203117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.203129 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:02:51.203167 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.203169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.203187 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.203464 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:51.203570 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.204580 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.206898 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.207700 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.208258 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.208369 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.208398 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.208440 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13387,"thread_start_us":69,"threads_started":1}
I20260808 07:02:51.210597 11772 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 (DEFAULT_TABLE table=test-table [id=0c4fed1b4e124c308c623ecbe717ada8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:51.211035 11772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3f98f1c7ee47cf9007cc8496936815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:51.211592 11920 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: No bootstrap required, opened a new log
I20260808 07:02:51.211673 11920 ts_tablet_manager.cc:1397] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260808 07:02:51.214113 11920 raft_consensus.cc:348] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.214298 11920 raft_consensus.cc:374] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:51.214346 11920 raft_consensus.cc:729] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386d5ae613714808acde3fb54196c344, State: Initialized, Role: FOLLOWER
I20260808 07:02:51.214493 11920 consensus_queue.cc:260] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.214758 11920 ts_tablet_manager.cc:1428] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:02:51.214948 11665 heartbeater.cc:503] Master 127.24.181.126:33813 was elected leader, sending a full tablet report...
I20260808 07:02:51.217195 11923 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Bootstrap starting.
I20260808 07:02:51.217850 11923 tablet_bootstrap.cc:654] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:51.218140 11923 log.cc:824] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:51.218503 11924 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Bootstrap starting.
I20260808 07:02:51.219120 11924 tablet_bootstrap.cc:654] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:51.219434 11923 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: No bootstrap required, opened a new log
I20260808 07:02:51.219570 11923 ts_tablet_manager.cc:1397] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:51.219568 11924 log.cc:824] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:51.220355 11924 tablet_bootstrap.cc:492] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: No bootstrap required, opened a new log
I20260808 07:02:51.220435 11924 ts_tablet_manager.cc:1397] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:51.222669 11923 raft_consensus.cc:348] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.222904 11923 raft_consensus.cc:374] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:51.223024 11923 raft_consensus.cc:729] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5304bea8f60144b99e3904f029ba08e0, State: Initialized, Role: FOLLOWER
I20260808 07:02:51.222997 11924 raft_consensus.cc:348] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.223142 11924 raft_consensus.cc:374] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:51.223184 11924 raft_consensus.cc:729] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b205656263147dbb5198925aaeeca07, State: Initialized, Role: FOLLOWER
I20260808 07:02:51.223308 11924 consensus_queue.cc:260] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.223343 11923 consensus_queue.cc:260] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.223577 11924 ts_tablet_manager.cc:1428] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 07:02:51.223718 11873 heartbeater.cc:503] Master 127.24.181.126:33813 was elected leader, sending a full tablet report...
I20260808 07:02:51.224192 11923 ts_tablet_manager.cc:1428] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260808 07:02:51.224306 11487 heartbeater.cc:503] Master 127.24.181.126:33813 was elected leader, sending a full tablet report...
W20260808 07:02:51.237540 11488 tablet.cc:2369] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:51.282685 11932 raft_consensus.cc:482] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:51.282868 11932 raft_consensus.cc:504] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.283365 11932 leader_election.cc:288] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 386d5ae613714808acde3fb54196c344 (127.24.181.66:41941), 9b205656263147dbb5198925aaeeca07 (127.24.181.67:36517)
I20260808 07:02:51.287536 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.283764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:51331 (local address 127.24.181.66:41941)
0808 07:02:51.283812 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.283814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.283998 (+ 184us) server_negotiation.cc:407] Connection header received
0808 07:02:51.284095 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.284096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.284106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.284133 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:51.284359 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.285061 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.286544 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.287097 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.287150 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.287158 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.287413 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.287453 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:02:51.287575 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.283835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:60193 (local address 127.24.181.67:36517)
0808 07:02:51.283880 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.283882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.284009 (+ 127us) server_negotiation.cc:407] Connection header received
0808 07:02:51.284118 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.284120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.284130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.284177 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:51.284418 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.285150 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.286550 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.287133 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.287195 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.287203 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.287448 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.287493 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:02:51.287842 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.283612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:36517 (local address 127.24.181.65:60193)
0808 07:02:51.283941 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.283952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:51.284036 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:51.284251 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:51.284252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:51.284255 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:51.284333 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.284335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.285258 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.285259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.286440 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.286443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.287265 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.287266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.287354 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.287359 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:51.287706 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 07:02:51.287752 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":98,"threads_started":1}
I20260808 07:02:51.288175 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.283527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:41941 (local address 127.24.181.65:51331)
0808 07:02:51.283934 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.283945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:51.284025 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:51.284198 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:51.284199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:51.284201 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:51.284288 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.284290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.285181 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.285182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.286430 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:51.286433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.287242 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.287243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:51.287325 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.287329 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:51.288045 (+ 716us) client_negotiation.cc:240] Negotiation successful
0808 07:02:51.288094 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":117,"threads_started":1}
I20260808 07:02:51.288193 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3f98f1c7ee47cf9007cc8496936815" candidate_uuid: "5304bea8f60144b99e3904f029ba08e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b205656263147dbb5198925aaeeca07" is_pre_election: true
I20260808 07:02:51.288355 11804 raft_consensus.cc:2454] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5304bea8f60144b99e3904f029ba08e0 in term 0.
I20260808 07:02:51.288530 11586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3f98f1c7ee47cf9007cc8496936815" candidate_uuid: "5304bea8f60144b99e3904f029ba08e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "386d5ae613714808acde3fb54196c344" is_pre_election: true
I20260808 07:02:51.288599 11335 leader_election.cc:302] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5304bea8f60144b99e3904f029ba08e0, 9b205656263147dbb5198925aaeeca07; no voters:
I20260808 07:02:51.288676 11586 raft_consensus.cc:2454] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5304bea8f60144b99e3904f029ba08e0 in term 0.
I20260808 07:02:51.288863 11932 raft_consensus.cc:2781] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:51.288939 11932 raft_consensus.cc:482] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:51.288964 11932 raft_consensus.cc:3037] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:51.289949 11932 raft_consensus.cc:504] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.290108 11932 leader_election.cc:288] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [CANDIDATE]: Term 1 election: Requested vote from peers 386d5ae613714808acde3fb54196c344 (127.24.181.66:41941), 9b205656263147dbb5198925aaeeca07 (127.24.181.67:36517)
I20260808 07:02:51.290295 11586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3f98f1c7ee47cf9007cc8496936815" candidate_uuid: "5304bea8f60144b99e3904f029ba08e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "386d5ae613714808acde3fb54196c344"
I20260808 07:02:51.290313 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3f98f1c7ee47cf9007cc8496936815" candidate_uuid: "5304bea8f60144b99e3904f029ba08e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b205656263147dbb5198925aaeeca07"
I20260808 07:02:51.290372 11586 raft_consensus.cc:3037] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:51.290390 11804 raft_consensus.cc:3037] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:51.291323 11804 raft_consensus.cc:2454] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5304bea8f60144b99e3904f029ba08e0 in term 1.
I20260808 07:02:51.291502 11586 raft_consensus.cc:2454] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5304bea8f60144b99e3904f029ba08e0 in term 1.
I20260808 07:02:51.291508 11335 leader_election.cc:302] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5304bea8f60144b99e3904f029ba08e0, 9b205656263147dbb5198925aaeeca07; no voters:
I20260808 07:02:51.291586 11932 raft_consensus.cc:2781] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:51.291733 11932 raft_consensus.cc:686] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [term 1 LEADER]: Becoming Leader. State: Replica: 5304bea8f60144b99e3904f029ba08e0, State: Running, Role: LEADER
I20260808 07:02:51.291822 11932 consensus_queue.cc:237] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } }
I20260808 07:02:51.292651 11217 catalog_manager.cc:5495] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5304bea8f60144b99e3904f029ba08e0 (127.24.181.65). New cstate: current_term: 1 leader_uuid: "5304bea8f60144b99e3904f029ba08e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5304bea8f60144b99e3904f029ba08e0" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 36861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:51.309278 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.305842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.24.181.65:36861)
0808 07:02:51.305887 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:51.305889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:51.305899 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:51.305945 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:51.305946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:51.305967 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:51.306043 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:51.306268 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.307004 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.308092 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:51.308384 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:51.308449 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:51.308699 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:51.308845 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:51.308925 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:51.309178 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 07:02:51.309199 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:51.311563 11586 raft_consensus.cc:1261] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 1 FOLLOWER]: Refusing update from remote peer 5304bea8f60144b99e3904f029ba08e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:51.311571 11804 raft_consensus.cc:1261] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 1 FOLLOWER]: Refusing update from remote peer 5304bea8f60144b99e3904f029ba08e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:51.311980 11937 consensus_queue.cc:1035] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b205656263147dbb5198925aaeeca07" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:51.312224 11932 consensus_queue.cc:1035] T 9f3f98f1c7ee47cf9007cc8496936815 P 5304bea8f60144b99e3904f029ba08e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "386d5ae613714808acde3fb54196c344" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 41941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:51.317353 11944 mvcc.cc:205] Tried to move back new op lower bound from 7316162852088659968 to 7316162852011896832. Current Snapshot: MvccSnapshot[applied={T|T < 7316162852088659968}]
I20260808 07:02:51.317902 11947 mvcc.cc:205] Tried to move back new op lower bound from 7316162852088659968 to 7316162852011896832. Current Snapshot: MvccSnapshot[applied={T|T < 7316162852088659968}]
I20260808 07:02:51.320441 11217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:51.320544 11217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:51.322288 11217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dda19c98fa8e41c19707f7191a7d226a: Sending DeleteTablet for 3 replicas of tablet 9f3f98f1c7ee47cf9007cc8496936815
I20260808 07:02:51.322675 11397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:51 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:46458
I20260808 07:02:51.323169 11563 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:51 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:42916
I20260808 07:02:51.323457 11772 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3f98f1c7ee47cf9007cc8496936815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:51 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:50068
I20260808 07:02:51.323452 11945 mvcc.cc:205] Tried to move back new op lower bound from 7316162852088659968 to 7316162852011896832. Current Snapshot: MvccSnapshot[applied={T|T < 7316162852088659968}]
I20260808 07:02:51.324075 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 11268
I20260808 07:02:51.330621 11959 tablet_replica.cc:330] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: stopping tablet replica
I20260808 07:02:51.330828 11959 raft_consensus.cc:2229] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:51.330998 11959 raft_consensus.cc:2258] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:51.336431 11956 tablet_replica.cc:330] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: stopping tablet replica
I20260808 07:02:51.336577 11956 raft_consensus.cc:2229] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:51.336741 11956 raft_consensus.cc:2258] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:51.337695 11956 ts_tablet_manager.cc:1905] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:51.338227 11959 ts_tablet_manager.cc:1905] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:51.339989 11959 ts_tablet_manager.cc:1918] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:51.340165 11959 log.cc:1196] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/9f3f98f1c7ee47cf9007cc8496936815
I20260808 07:02:51.340555 11959 ts_tablet_manager.cc:1939] T 9f3f98f1c7ee47cf9007cc8496936815 P 9b205656263147dbb5198925aaeeca07: Deleting consensus metadata
I20260808 07:02:51.341081 11956 ts_tablet_manager.cc:1918] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:51.341158 11956 log.cc:1196] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/9f3f98f1c7ee47cf9007cc8496936815
I20260808 07:02:51.341137 11198 catalog_manager.cc:4841] TS 9b205656263147dbb5198925aaeeca07 (127.24.181.67:36517): tablet 9f3f98f1c7ee47cf9007cc8496936815 (table test-table [id=0c4fed1b4e124c308c623ecbe717ada8]) successfully deleted
I20260808 07:02:51.341445 11956 ts_tablet_manager.cc:1939] T 9f3f98f1c7ee47cf9007cc8496936815 P 386d5ae613714808acde3fb54196c344: Deleting consensus metadata
I20260808 07:02:51.341962 11200 catalog_manager.cc:4841] TS 386d5ae613714808acde3fb54196c344 (127.24.181.66:41941): tablet 9f3f98f1c7ee47cf9007cc8496936815 (table test-table [id=0c4fed1b4e124c308c623ecbe717ada8]) successfully deleted
W20260808 07:02:51.344889 11199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:36861 (error 108)
I20260808 07:02:51.345291 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 11492
I20260808 07:02:51.362012 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:51.361852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:36861 (local address 127.0.0.1:46480)
0808 07:02:51.361915 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:51.361961 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 07:02:51.363060 11199 catalog_manager.cc:4584] TS 5304bea8f60144b99e3904f029ba08e0 (127.24.181.65:36861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f3f98f1c7ee47cf9007cc8496936815: Network error: Client connection negotiation failed: client connection to 127.24.181.65:36861: connect: Connection refused (error 111)
I20260808 07:02:51.367131 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 11669
Aug 08 07:02:51 dist-test-slave-nfb5 krb5kdc[9796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for HTTP/127.24.181.84@KRBTEST.COM
I20260808 07:02:51.382321 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 11174
I20260808 07:02:51.393486 25301 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:02:51.749836 25301 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:02:51.749943 25301 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:02:52.104218 25301 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:02:52.104 UTC [9836] LOG: received smart shutdown request
2026-08-08 07:02:52.109 UTC [9836] LOG: background worker "logical replication launcher" (PID 9848) exited with exit code 1
2026-08-08 07:02:52.109 UTC [9840] LOG: shutting down
2026-08-08 07:02:52.265 UTC [9836] LOG: database system is shut down
2026-08-08T07:02:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49956 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:52 dist-test-slave-nfb5 krb5kdc[11975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:52 dist-test-slave-nfb5 krb5kdc[11975](info): set up 2 sockets
Aug 08 07:02:52 dist-test-slave-nfb5 krb5kdc[11975](info): commencing operation
krb5kdc: starting...
W20260808 07:02:53.643774 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:02:53 dist-test-slave-nfb5 krb5kdc[11975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:02:53.770267 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41003
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:53.854174 11991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:53.854383 11991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:53.854406 11991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:53.855585 11991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:53.855628 11991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:53.855643 11991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:53.855655 11991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:53.855715 11991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:53.857191 11991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:53.857463 11991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:53.857698 11991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:53.862547 11997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:53.862581 11996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:53.862552 12001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:53.862787 11991 server_base.cc:1034] running on GCE node
I20260808 07:02:53.863065 11991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:53.863312 11991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:53.864476 11991 hybrid_clock.cc:648] HybridClock initialized: now 1786172573864479 us; error 69 us; skew 500 ppm
Aug 08 07:02:53 dist-test-slave-nfb5 krb5kdc[11975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:53.869746 11991 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:53.870172 11991 webserver.cc:460] Webserver started at http://127.24.181.126:36777/ using document root <none> and password file <none>
I20260808 07:02:53.870398 11991 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:53.870466 11991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:53.870618 11991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:53.871747 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c0536c7eecf24d09b2fccb8d50bb1c06"
format_stamp: "Formatted at 2026-08-08 07:02:53 on dist-test-slave-nfb5"
server_key: "be199ed8b73368e1ba9f5eeb5c1a23d0"
server_key_iv: "0b1092f35a306b6d91e2eb37d4ad0d2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:53.872112 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c0536c7eecf24d09b2fccb8d50bb1c06"
format_stamp: "Formatted at 2026-08-08 07:02:53 on dist-test-slave-nfb5"
server_key: "be199ed8b73368e1ba9f5eeb5c1a23d0"
server_key_iv: "0b1092f35a306b6d91e2eb37d4ad0d2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:53.873441 11991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:53.874179 12012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:53.874351 11991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:53.874457 11991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c0536c7eecf24d09b2fccb8d50bb1c06"
format_stamp: "Formatted at 2026-08-08 07:02:53 on dist-test-slave-nfb5"
server_key: "be199ed8b73368e1ba9f5eeb5c1a23d0"
server_key_iv: "0b1092f35a306b6d91e2eb37d4ad0d2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:53.874533 11991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:53.895505 11991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:53.897990 11991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:53.898165 11991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:53.902488 11991 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:41003
I20260808 07:02:53.902525 12095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:41003 every 8 connection(s)
I20260808 07:02:53.902938 11991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:53.903858 12096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:53.905501 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 11991
I20260808 07:02:53.905382 12096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:53.905637 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:02:53.905820 25301 external_mini_cluster.cc:1428] Setting key 9433b4f29d1942cb90b574c1763009fa
Aug 08 07:02:53 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.125@KRBTEST.COM; defaulting to no policy
I20260808 07:02:53.916363 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.906098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41003 (local address 127.0.0.1:37132)
0808 07:02:53.906316 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:53.906359 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:53.907131 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:53.907426 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:53.907429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:53.907807 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:53.907969 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:53.907974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:53.909267 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:53.909268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:53.909947 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:53.909949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:53.910694 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:53.910696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:53.910782 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:53.911312 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:53.911320 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:53.913072 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:53.914886 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:53.914895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:53.914897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:53.915193 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:53.915430 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:53.915431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:53.915431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:53.915516 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:53.915877 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:53.915883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:53.915952 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 07:02:53.916020 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:02:53.916142 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260808 07:02:53.916368 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.906151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.24.181.126:41003)
0808 07:02:53.906337 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:53.906347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:53.906454 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:02:53.907281 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:53.907283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:53.907308 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:53.907389 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:53.908084 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:53.909169 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:53.910037 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:53.910591 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:53.910711 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:53.913163 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:53.913170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:53.913171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:53.913177 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:53.914774 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:53.915279 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:53.915282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:53.915282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:53.915362 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:53.915580 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:53.915581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:53.915581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:53.915807 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:53.915872 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:53.916097 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 07:02:53.916208 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 07:02:53.917147 12096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:53.917484 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.917379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54944)
0808 07:02:53.917425 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:53.917467 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:02:53.917637 12019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.125@KRBTEST.COM" created.
I20260808 07:02:53.918136 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.918019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54950)
0808 07:02:53.918081 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:53.918112 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:02:53.918268 12096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:43041: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
I20260808 07:02:53.931630 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:43041
--webserver_interface=127.24.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:53.967511 12096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } attempt: 1
I20260808 07:02:53.968130 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.967947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54964)
0808 07:02:53.968025 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:53.968082 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:02:53.968578 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:53.968448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54980)
0808 07:02:53.968504 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:53.968542 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:02:53.968727 12096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:43041: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
W20260808 07:02:54.014564 12104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:54.014773 12104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:54.014796 12104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:54.015975 12104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:54.016022 12104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:54.016037 12104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:54.016050 12104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:54.016062 12104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:54.017581 12104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:43041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:54.017864 12104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:54.018095 12104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:54.022984 12110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:54.023029 12109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:54.022964 12112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:54.023175 12104 server_base.cc:1034] running on GCE node
I20260808 07:02:54.023415 12104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:54.023698 12104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:54.024879 12104 hybrid_clock.cc:648] HybridClock initialized: now 1786172574024864 us; error 28 us; skew 500 ppm
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:54.027468 12104 init.cc:377] Logged in from keytab as kudu/127.24.181.125@KRBTEST.COM (short username kudu)
I20260808 07:02:54.027938 12104 webserver.cc:460] Webserver started at http://127.24.181.125:34339/ using document root <none> and password file <none>
I20260808 07:02:54.028189 12104 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:54.028260 12104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:54.028406 12104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:54.029452 12104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data/instance:
uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "9f8878394b2023120e6f5ebaf25d22cd"
server_key_iv: "271702881efba45be74f8ac763e35792"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.029827 12104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal/instance:
uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "9f8878394b2023120e6f5ebaf25d22cd"
server_key_iv: "271702881efba45be74f8ac763e35792"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.031183 12104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:02:54.031885 12122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:54.032060 12104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:54.032152 12104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "9f8878394b2023120e6f5ebaf25d22cd"
server_key_iv: "271702881efba45be74f8ac763e35792"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.032236 12104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:54.036916 12096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } attempt: 2
I20260808 07:02:54.037289 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.037167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54986)
0808 07:02:54.037224 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.037260 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:02:54.037640 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.037560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:54998)
0808 07:02:54.037599 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.037623 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 07:02:54.037742 12096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:43041: Network error: Client connection negotiation failed: client connection to 127.24.181.125:43041: connect: Connection refused (error 111)
I20260808 07:02:54.049403 12104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:54.051360 12104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:54.051522 12104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:54.055773 12104 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.125:43041
I20260808 07:02:54.055838 12206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.125:43041 every 8 connection(s)
I20260808 07:02:54.056174 12104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
I20260808 07:02:54.056625 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12104
I20260808 07:02:54.056771 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-1/wal/instance
I20260808 07:02:54.056973 25301 external_mini_cluster.cc:1428] Setting key b5a25213610a093824457490d87708e7
I20260808 07:02:54.057271 12207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:54.059566 12207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.181.124@KRBTEST.COM; defaulting to no policy
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.124@KRBTEST.COM" created.
I20260808 07:02:54.068835 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.060295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.24.181.126:41003)
0808 07:02:54.060370 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.060373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.060553 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:02:54.061056 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.061057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.061070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.061134 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.061754 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.062253 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.063301 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.063516 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.063596 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.065909 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.065911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.065911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.065918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.067364 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.067755 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.067756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.067757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.067847 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.068128 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.068129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.068129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.068284 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.068358 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.068687 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.068758 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:02:54.069147 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.060266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41003 (local address 127.0.0.1:37146)
0808 07:02:54.060438 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.060474 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.060953 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.061208 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.061211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.061459 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.061644 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.061648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.062354 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.062355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.063211 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.063214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.063603 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.063604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.063728 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.064265 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.064272 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.065824 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.067434 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.067441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.067443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.067660 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.067916 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.067916 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.067917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.068053 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.068361 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.068369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.068545 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.068664 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.068850 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260808 07:02:54.070091 12207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:02:54.077682 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.070382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:55006)
0808 07:02:54.070430 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.070440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.070500 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.070865 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.070866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.071033 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.071074 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.071076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.071916 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.071917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.072417 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.072419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.072878 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.072879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.072946 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.073398 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.073402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.074752 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.076551 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.076552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.076552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.076716 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.076979 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.076979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.076980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.077036 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.077335 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.077335 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.077375 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.077442 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.077549 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:02:54.077809 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.070506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.24.181.125:43041)
0808 07:02:54.070694 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.070699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.070723 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:02:54.070746 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.070748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.070762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.070866 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.071153 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.071814 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.072499 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.072784 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.072879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.074851 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.074860 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.074860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.074867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.076429 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.076810 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.076813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.076814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.076880 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.077111 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.077112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.077112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.077258 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.077352 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.077489 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.077622 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260808 07:02:54.078512 12207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:54.078871 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.078740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37490)
0808 07:02:54.078789 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.078841 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:02:54.079054 12126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
I20260808 07:02:54.079598 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.079506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37500)
0808 07:02:54.079554 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.079580 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:02:54.079768 12207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:39713: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
I20260808 07:02:54.085807 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:39713
--webserver_interface=127.24.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:54.128980 12096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } attempt: 3
I20260808 07:02:54.129007 12207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } attempt: 1
I20260808 07:02:54.129555 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.129358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37502)
0808 07:02:54.129461 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.129510 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 07:02:54.130184 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.130009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37506)
0808 07:02:54.130114 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.130158 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260808 07:02:54.130282 12207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:39713: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:02:54.136874 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.129361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:55010)
0808 07:02:54.129425 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.129436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.129483 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.129647 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.129648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.129893 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.129996 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.129999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.130810 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.130811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.131396 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.131398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.131779 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.131780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.131867 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.132392 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.132396 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.133894 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.135922 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.135923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.135923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.136086 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.136308 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.136308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.136308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.136365 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.136651 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.136652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.136689 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.136736 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.136810 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:02:54.136946 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.129424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.24.181.125:43041)
0808 07:02:54.129472 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.129474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.129484 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:54.129540 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.129541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.129554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.129636 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.130074 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.130718 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.131485 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.131702 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.131782 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.133994 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.133996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.133996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.134005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.135833 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.136165 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.136167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.136167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.136237 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.136428 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.136428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.136429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.136579 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.136661 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.136767 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.136853 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:02:54.137530 12096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:54.137893 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.137765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37514)
0808 07:02:54.137819 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.137859 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:02:54.138192 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.138125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37528)
0808 07:02:54.138149 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.138174 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 07:02:54.138293 12096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:39713: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
I20260808 07:02:54.169457 12096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } attempt: 1
I20260808 07:02:54.169991 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.169833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37532)
0808 07:02:54.169900 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.169948 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:02:54.170400 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.170279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37548)
0808 07:02:54.170334 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.170369 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:02:54.170533 12096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:39713: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
W20260808 07:02:54.180855 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:54.181057 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:54.181082 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:54.182317 12218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:54.182356 12218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:54.182372 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:54.182386 12218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:54.182400 12218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:54.184063 12218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41003,127.24.181.125:43041,127.24.181.124:39713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:54.184273 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:54.184473 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:54.189323 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:54.189296 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:54.189288 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:54.189530 12218 server_base.cc:1034] running on GCE node
I20260808 07:02:54.189823 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:54.190129 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:54.191290 12218 hybrid_clock.cc:648] HybridClock initialized: now 1786172574191260 us; error 52 us; skew 500 ppm
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:54.193727 12218 init.cc:377] Logged in from keytab as kudu/127.24.181.124@KRBTEST.COM (short username kudu)
I20260808 07:02:54.194185 12218 webserver.cc:460] Webserver started at http://127.24.181.124:38549/ using document root <none> and password file <none>
I20260808 07:02:54.194396 12218 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:54.194437 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:54.194527 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:54.195590 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data/instance:
uuid: "0b2b4d3b510b4b82b199fe670de4c80d"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "2eaa99d0fc4579cd211339b7fed39264"
server_key_iv: "530c42ff595284c673ec7dc1ff7af106"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.195901 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal/instance:
uuid: "0b2b4d3b510b4b82b199fe670de4c80d"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "2eaa99d0fc4579cd211339b7fed39264"
server_key_iv: "530c42ff595284c673ec7dc1ff7af106"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.197175 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:54.197923 12240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:54.198112 12218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:54.198186 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "0b2b4d3b510b4b82b199fe670de4c80d"
format_stamp: "Formatted at 2026-08-08 07:02:54 on dist-test-slave-nfb5"
server_key: "2eaa99d0fc4579cd211339b7fed39264"
server_key_iv: "530c42ff595284c673ec7dc1ff7af106"
server_key_version: "encryptionkey@0"
I20260808 07:02:54.198288 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:54.198413 12207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } attempt: 2
I20260808 07:02:54.198820 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.198665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37550)
0808 07:02:54.198735 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.198778 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:02:54.199218 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.199105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37558)
0808 07:02:54.199155 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.199191 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:02:54.199318 12207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:39713: Network error: Client connection negotiation failed: client connection to 127.24.181.124:39713: connect: Connection refused (error 111)
I20260808 07:02:54.237736 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:54.239704 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:54.239868 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:54.244099 12218 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.124:39713
I20260808 07:02:54.244171 12316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.124:39713 every 8 connection(s)
I20260808 07:02:54.244529 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
I20260808 07:02:54.245414 12317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:54.245730 12096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } attempt: 2
I20260808 07:02:54.247860 12317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:02:54.251881 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12218
I20260808 07:02:54.252007 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172407597382-25301-0/minicluster-data/master-2/wal/instance
I20260808 07:02:54.252460 25301 external_mini_cluster.cc:1428] Setting key 0480b3fad66f53e70b39139dd4f9b84e
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:54.255357 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.246053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37568)
0808 07:02:54.246121 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.246134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.246200 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.247205 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.247206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.247400 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.247459 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.247462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.248811 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.248812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.249319 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.249321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.249958 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.249959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.250014 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.250499 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.250504 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.251733 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.253817 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.253818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.253818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.254069 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.254377 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.254378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.254379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.254453 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.254882 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.254883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.254956 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.255118 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.255236 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:02:54.255769 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.246230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.24.181.124:39713)
0808 07:02:54.246408 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.246416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.246440 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:02:54.247004 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.247013 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.247074 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.247190 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.247566 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.248711 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.249402 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.249869 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.249994 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.251820 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.251829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.251829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.251842 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.253714 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.254166 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.254169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.254169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.254279 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.254526 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.254526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.254527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.254797 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.254874 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.255193 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.255321 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260808 07:02:54.256735 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.248515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.24.181.126:41003)
0808 07:02:54.248568 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.248570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.248727 (+ 157us) server_negotiation.cc:407] Connection header received
0808 07:02:54.248804 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.248805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.248816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.248889 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.249336 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.249794 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.250600 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.250789 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.250880 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.253418 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.253419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.253419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.253426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.255245 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.255697 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.255699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.255699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.255761 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.255971 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.255972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.255973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.256082 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.256164 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.256547 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.256640 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":32}
I20260808 07:02:54.256759 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.248469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41003 (local address 127.0.0.1:37160)
0808 07:02:54.248632 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.248664 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.248745 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.248912 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.248915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.249178 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.249243 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.249244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.249883 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.249884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.250510 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.250512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.250895 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.250896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.251000 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.251474 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.251501 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.253310 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.255325 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.255332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.255334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.255620 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.255839 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.255839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.255840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.255899 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.256349 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.256352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.256401 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.256500 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.256621 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":37,"thread_start_us":79,"threads_started":1}
I20260808 07:02:54.257246 12317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:02:54.258394 12096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: Bootstrap starting.
I20260808 07:02:54.259299 12096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:54.259656 12096 log.cc:824] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:54.260732 12096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: No bootstrap required, opened a new log
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:02:54.262224 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.253044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.24.181.124:39713)
0808 07:02:54.253186 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.253188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.253333 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:02:54.253406 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.253407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.253417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.253483 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.254257 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.254963 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.256000 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.256257 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.256310 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.258884 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.258886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.258886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.258893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.260737 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.261305 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.261306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.261307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.261371 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.261580 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.261581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.261581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.261689 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.261842 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.261920 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.261998 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260808 07:02:54.263306 12096 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.263448 12096 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:54.263491 12096 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0536c7eecf24d09b2fccb8d50bb1c06, State: Initialized, Role: FOLLOWER
I20260808 07:02:54.263622 12096 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.263967 12096 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:54.264464 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.264571 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.265623 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.257532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:43041 (local address 127.0.0.1:55014)
0808 07:02:54.257578 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.257588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.257637 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.257783 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.257784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.257941 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.257996 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.257999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.258877 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.258878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.260071 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.260074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.260580 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.260580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.260665 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.261031 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.261036 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.262458 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.264484 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.264485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.264486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.264725 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.264939 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.264940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.264940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.264999 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.265324 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.265325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.265371 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.265438 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.265534 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:02:54.265641 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.257616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.24.181.125:43041)
0808 07:02:54.257660 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.257662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.257671 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:54.257696 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.257697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.257707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.257768 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.258096 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.258770 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.260162 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.260487 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.260571 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.262557 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.262559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.262559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.262567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.264279 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.264819 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.264821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.264821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.264885 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.265103 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.265104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.265104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.265257 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.265320 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.265478 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.265554 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:02:54.266196 12317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
W20260808 07:02:54.267037 12343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:54.267117 12343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:54.266790 11991 master_runner.cc:186] Error getting master registration for 127.24.181.125:43041: OK, instance_id { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" instance_seqno: 1786172574054486 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:02:54.269500 11991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:02:54.271968 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.263125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.24.181.126:41003)
0808 07:02:54.263165 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.263167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.263175 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:54.263245 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.263246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.263256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.263310 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.264123 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.264740 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.265779 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.266040 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.266111 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.268751 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.268752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.268752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.268759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.270413 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.270805 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.270806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.270807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.270873 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.271123 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.271124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.271124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.271261 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.271410 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.271826 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.271912 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:02:54.275954 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.266673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37584)
0808 07:02:54.266890 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.266900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.266969 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.267319 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.267320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.267471 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.267503 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.267505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.270052 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.270054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.270735 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.270738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.271277 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.271278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.271344 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.271749 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.271754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.273009 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.274748 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.274749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.274749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.275014 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.275237 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.275237 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.275238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.275293 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.275639 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.275640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.275686 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.275759 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.275855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260808 07:02:54.276209 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.266768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.24.181.124:39713)
0808 07:02:54.267186 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.267188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.267199 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:54.267220 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.267221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.267233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.267302 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.267596 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.269961 (+ 2365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.270845 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.271183 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.271261 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.273108 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.273109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.273110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.273115 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.274658 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.275101 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.275103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.275103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.275166 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.275366 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.275367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.275367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.275545 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.275639 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.276062 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.276132 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:02:54.277864 12317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: Bootstrap starting.
I20260808 07:02:54.278609 12317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:54.278945 12317 log.cc:824] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:54.280725 12317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: No bootstrap required, opened a new log
I20260808 07:02:54.281071 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.272698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.24.181.125:43041)
0808 07:02:54.272770 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.272772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.272783 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:54.272867 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.272868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.272878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.272922 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.273532 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.274214 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.275161 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.275459 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.275537 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.278185 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.278187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.278187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.278195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.279786 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.280197 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.280198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.280198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.280264 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.280474 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.280476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.280476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.280598 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.280854 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.280898 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.280973 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:02:54.282312 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:02:54.282709 12317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.282867 12317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:54.282902 12317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b4d3b510b4b82b199fe670de4c80d, State: Initialized, Role: FOLLOWER
I20260808 07:02:54.283089 12317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.283406 12317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:54.283425 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.283598 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.286803 12218 master_runner.cc:186] Error getting master registration for 127.24.181.125:43041: OK, instance_id { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" instance_seqno: 1786172574054486 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:02:54.289518 12218 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:02:54.290436 12207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } attempt: 3
W20260808 07:02:54.291455 12374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:54.291507 12374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:02:54.295275 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.284419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.24.181.124:39713)
0808 07:02:54.284656 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.284658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.287590 (+ 2932us) server_negotiation.cc:407] Connection header received
0808 07:02:54.287678 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.287679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.287688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.287716 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.288419 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.289413 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.290473 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.290790 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.290860 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.292255 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.292257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.292257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.292264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.293951 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.294338 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.294339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.294339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.294391 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.294579 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.294580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.294581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.294694 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.294777 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.295115 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.295185 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
Aug 08 07:02:54 dist-test-slave-nfb5 krb5kdc[11975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:02:54.298148 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.283897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.24.181.126:41003)
0808 07:02:54.283936 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.283938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.284765 (+ 827us) server_negotiation.cc:407] Connection header received
0808 07:02:54.284892 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.284893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.284904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.284957 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.285597 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.286134 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.287163 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.287475 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.287517 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.292187 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.292188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.292188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.292195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.294177 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.296429 (+ 2252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.296430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.296430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.296494 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.296975 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.296977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.296977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.297120 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.297932 (+ 812us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.297980 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.298062 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:02:54.299261 12212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.290662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:39713 (local address 127.0.0.1:37594)
0808 07:02:54.290753 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:54.290763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:54.290799 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:54.291111 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:54.291111 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:54.291280 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:54.291321 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.291323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.292028 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.292028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.293820 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:54.293823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.294402 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.294404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:54.294466 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.294897 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:54.294902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:54.296430 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:54.298209 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.298210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.298211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.298430 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.298628 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:54.298628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:54.298629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:54.298684 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:54.298964 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:54.298965 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:54.299009 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:02:54.299073 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:02:54.299170 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":2}
I20260808 07:02:54.299549 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.290944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.24.181.124:39713)
0808 07:02:54.290982 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.290984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.290991 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:02:54.291009 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.291010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.291019 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.291068 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.291407 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.291931 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.293930 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.294289 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.294350 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.296525 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.296526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.296526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.296533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.298115 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.298511 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.298512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.298512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.298563 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.298747 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.298748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.298748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.298858 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.298937 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.299384 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.299464 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:02:54.300213 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:54.284088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.24.181.125:43041)
0808 07:02:54.284125 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:54.284127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:54.284368 (+ 241us) server_negotiation.cc:407] Connection header received
0808 07:02:54.284505 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:54.284506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:54.284517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:54.284551 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:54.287574 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.288302 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.292488 (+ 4186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:54.292792 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:54.292837 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:54.295558 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:54.295560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:54.295560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:54.295567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:54.297438 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.298738 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.298739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.298739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.298800 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:54.299020 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:54.299021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:54.299021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:54.299150 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:54.299804 (+ 654us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:54.300048 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 07:02:54.300125 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260808 07:02:54.301362 12207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: Bootstrap starting.
I20260808 07:02:54.302062 12207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:54.302428 12207 log.cc:824] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:54.303360 12207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: No bootstrap required, opened a new log
I20260808 07:02:54.305416 12207 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.305540 12207 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:54.305570 12207 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2cfa6adff4421bbb35538e5bcb7fe7, State: Initialized, Role: FOLLOWER
I20260808 07:02:54.305756 12207 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.306090 12207 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:54.307106 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.307264 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.310074 12104 master_runner.cc:263] No leader master found from master bf2cfa6adff4421bbb35538e5bcb7fe7
I20260808 07:02:54.310128 12104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 07:02:54.310531 12390 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:54.310586 12390 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 07:02:54.399044 12023 tablet.cc:2369] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:54.490531 12249 tablet.cc:2369] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:54.552327 12130 tablet.cc:2369] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:54.563570 12327 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:54.563699 12327 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.564042 12327 leader_election.cc:288] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf2cfa6adff4421bbb35538e5bcb7fe7 (127.24.181.125:43041), 0b2b4d3b510b4b82b199fe670de4c80d (127.24.181.124:39713)
I20260808 07:02:54.564322 12283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" is_pre_election: true
I20260808 07:02:54.564363 12159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" is_pre_election: true
I20260808 07:02:54.564528 12283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0536c7eecf24d09b2fccb8d50bb1c06 in term 0.
I20260808 07:02:54.564543 12159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0536c7eecf24d09b2fccb8d50bb1c06 in term 0.
I20260808 07:02:54.564780 12017 leader_election.cc:302] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b4d3b510b4b82b199fe670de4c80d, c0536c7eecf24d09b2fccb8d50bb1c06; no voters:
I20260808 07:02:54.564946 12327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:54.565023 12327 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:54.565042 12327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:54.565762 12327 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.566023 12327 leader_election.cc:288] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [CANDIDATE]: Term 1 election: Requested vote from peers bf2cfa6adff4421bbb35538e5bcb7fe7 (127.24.181.125:43041), 0b2b4d3b510b4b82b199fe670de4c80d (127.24.181.124:39713)
I20260808 07:02:54.566236 12159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7"
I20260808 07:02:54.566241 12283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b4d3b510b4b82b199fe670de4c80d"
I20260808 07:02:54.566329 12159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:54.566355 12283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:54.567162 12159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0536c7eecf24d09b2fccb8d50bb1c06 in term 1.
I20260808 07:02:54.567206 12283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0536c7eecf24d09b2fccb8d50bb1c06 in term 1.
I20260808 07:02:54.567349 12019 leader_election.cc:302] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf2cfa6adff4421bbb35538e5bcb7fe7, c0536c7eecf24d09b2fccb8d50bb1c06; no voters:
I20260808 07:02:54.567476 12327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:54.567633 12327 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [term 1 LEADER]: Becoming Leader. State: Replica: c0536c7eecf24d09b2fccb8d50bb1c06, State: Running, Role: LEADER
I20260808 07:02:54.567739 12327 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } }
I20260808 07:02:54.568122 12394 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0536c7eecf24d09b2fccb8d50bb1c06. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.568208 12394 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:54.568451 12397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:54.568660 12397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:54.570025 12283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [term 1 FOLLOWER]: Refusing update from remote peer c0536c7eecf24d09b2fccb8d50bb1c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:54.570307 12159 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [term 1 FOLLOWER]: Refusing update from remote peer c0536c7eecf24d09b2fccb8d50bb1c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:54.570333 12394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:54.570578 12394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:54.571475 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0536c7eecf24d09b2fccb8d50bb1c06. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.571579 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.572489 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0536c7eecf24d09b2fccb8d50bb1c06. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.572588 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.572942 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.573026 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:54.573130 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.573197 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:54.573184 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.573266 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.573356 12397 catalog_manager.cc:1347] Generated new cluster ID: cfbebd181b0e445ea635c4b88c21cd9d
I20260808 07:02:54.573403 12397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:54.573375 12399 mvcc.cc:205] Tried to move back new op lower bound from 7316162865436467200 to 7316162865430052864. Current Snapshot: MvccSnapshot[applied={T|T < 7316162865436467200}]
I20260808 07:02:54.573809 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0536c7eecf24d09b2fccb8d50bb1c06" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41003 } } peers { permanent_uuid: "bf2cfa6adff4421bbb35538e5bcb7fe7" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 43041 } } peers { permanent_uuid: "0b2b4d3b510b4b82b199fe670de4c80d" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 39713 } } }
I20260808 07:02:54.573895 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:54.573954 12402 mvcc.cc:205] Tried to move back new op lower bound from 7316162865436467200 to 7316162865430052864. Current Snapshot: MvccSnapshot[applied={T|T < 7316162865436467200 or (T in {7316162865436467200})}]
I20260808 07:02:54.619115 12397 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:54.619772 12397 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:54.624131 12397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0536c7eecf24d09b2fccb8d50bb1c06: Generated new TSK 0
I20260808 07:02:54.624413 12397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:02:55.291981 12374 catalog_manager.cc:1259] Loaded cluster ID: cfbebd181b0e445ea635c4b88c21cd9d
I20260808 07:02:55.292065 12374 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: loading cluster ID for follower catalog manager: success
I20260808 07:02:55.293025 12374 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: acquiring CA information for follower catalog manager: success
I20260808 07:02:55.293360 12374 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b2b4d3b510b4b82b199fe670de4c80d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:02:55.311126 12390 catalog_manager.cc:1259] Loaded cluster ID: cfbebd181b0e445ea635c4b88c21cd9d
I20260808 07:02:55.311456 12390 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: loading cluster ID for follower catalog manager: success
I20260808 07:02:55.312407 12390 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: acquiring CA information for follower catalog manager: success
I20260808 07:02:55.312740 12390 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf2cfa6adff4421bbb35538e5bcb7fe7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:02:55.672240 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 11991
I20260808 07:02:55.685774 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12104
I20260808 07:02:55.692975 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12218
2026-08-08T07:02:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3322 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:55 dist-test-slave-nfb5 krb5kdc[12422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:55 dist-test-slave-nfb5 krb5kdc[12422](info): set up 2 sockets
Aug 08 07:02:55 dist-test-slave-nfb5 krb5kdc[12422](info): commencing operation
krb5kdc: starting...
W20260808 07:02:56.913870 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:02:57.039021 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:39387
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:57.121943 12440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:57.122208 12440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:57.122251 12440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:57.123445 12440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:57.123513 12440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:57.123538 12440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:57.123561 12440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:57.123590 12440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:57.125097 12440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:39387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:57.125391 12440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:57.125633 12440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:57.130761 12447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:57.130757 12449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:57.130999 12440 server_base.cc:1034] running on GCE node
W20260808 07:02:57.131088 12446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:57.131369 12440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:57.131626 12440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:57.132817 12440 hybrid_clock.cc:648] HybridClock initialized: now 1786172577132791 us; error 45 us; skew 500 ppm
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172577, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:57.135462 12440 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:57.135942 12440 webserver.cc:460] Webserver started at http://127.24.181.126:32897/ using document root <none> and password file <none>
I20260808 07:02:57.136169 12440 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:57.136245 12440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:57.136399 12440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:57.137492 12440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c814865d384848138360ee579a680dcc"
format_stamp: "Formatted at 2026-08-08 07:02:57 on dist-test-slave-nfb5"
server_key: "4bcd515f5c6392445462c3a5a04f1d39"
server_key_iv: "a706cdcc0626bf2d13f8820a6f957693"
server_key_version: "encryptionkey@0"
I20260808 07:02:57.137858 12440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c814865d384848138360ee579a680dcc"
format_stamp: "Formatted at 2026-08-08 07:02:57 on dist-test-slave-nfb5"
server_key: "4bcd515f5c6392445462c3a5a04f1d39"
server_key_iv: "a706cdcc0626bf2d13f8820a6f957693"
server_key_version: "encryptionkey@0"
I20260808 07:02:57.139253 12440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:02:57.139957 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:57.140148 12440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:57.140270 12440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c814865d384848138360ee579a680dcc"
format_stamp: "Formatted at 2026-08-08 07:02:57 on dist-test-slave-nfb5"
server_key: "4bcd515f5c6392445462c3a5a04f1d39"
server_key_iv: "a706cdcc0626bf2d13f8820a6f957693"
server_key_version: "encryptionkey@0"
I20260808 07:02:57.140347 12440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:57.149473 12440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:57.151767 12440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:57.151925 12440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:57.156136 12440 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:39387
I20260808 07:02:57.156189 12537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:39387 every 8 connection(s)
I20260808 07:02:57.156544 12440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:57.157403 12538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:57.159787 12538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: Bootstrap starting.
I20260808 07:02:57.160389 12538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:57.160717 12538 log.cc:824] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:57.161432 12538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: No bootstrap required, opened a new log
I20260808 07:02:57.163091 12538 raft_consensus.cc:348] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } }
I20260808 07:02:57.163224 12538 raft_consensus.cc:374] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:57.163247 12538 raft_consensus.cc:729] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c814865d384848138360ee579a680dcc, State: Initialized, Role: FOLLOWER
I20260808 07:02:57.163410 12538 consensus_queue.cc:260] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } }
I20260808 07:02:57.163475 12538 raft_consensus.cc:388] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:57.163515 12538 raft_consensus.cc:482] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:57.163565 12538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:57.163968 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12440
I20260808 07:02:57.164086 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:02:57.164280 25301 external_mini_cluster.cc:1428] Setting key 61e77b757649b86e7e48e98f8a653713
I20260808 07:02:57.164296 12538 raft_consensus.cc:504] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } }
I20260808 07:02:57.164407 12538 leader_election.cc:302] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c814865d384848138360ee579a680dcc; no voters:
I20260808 07:02:57.164584 12538 leader_election.cc:288] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:57.164628 12542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:57.164803 12542 raft_consensus.cc:686] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [term 1 LEADER]: Becoming Leader. State: Replica: c814865d384848138360ee579a680dcc, State: Running, Role: LEADER
I20260808 07:02:57.164839 12538 sys_catalog.cc:564] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:57.164898 12542 consensus_queue.cc:237] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } }
I20260808 07:02:57.165305 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c814865d384848138360ee579a680dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } } }
I20260808 07:02:57.165436 12545 sys_catalog.cc:455] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c814865d384848138360ee579a680dcc. Latest consensus state: current_term: 1 leader_uuid: "c814865d384848138360ee579a680dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c814865d384848138360ee579a680dcc" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 39387 } } }
I20260808 07:02:57.165575 12545 sys_catalog.cc:458] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:57.165478 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc [sys.catalog]: This master's current role is: LEADER
W20260808 07:02:57.166664 12573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:57.166738 12573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:02:57.166815 12562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:57.167027 12562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:57.168701 12562 catalog_manager.cc:1347] Generated new cluster ID: fb3078efa1714f5dbbf8fad2335c2958
I20260808 07:02:57.168756 12562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:57.177930 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:57.164758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41528 (local address 127.24.181.126:39387)
0808 07:02:57.165869 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:57.165879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:57.165917 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:02:57.166036 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:57.166042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:57.166100 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:57.166261 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:57.167126 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:57.168333 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:57.169418 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:57.170039 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:57.170121 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:57.173177 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:57.173181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:57.173181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:57.173194 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:57.175855 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:57.176262 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:57.176265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:57.176266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:57.176387 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:57.176607 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:57.176608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:57.176608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:57.176919 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:57.177071 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:57.177568 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 07:02:57.177704 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":73,"threads_started":1}
I20260808 07:02:57.178762 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:02:57.190858 12562 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:57.191612 12562 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:57.197232 12562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c814865d384848138360ee579a680dcc: Generated new TSK 0
I20260808 07:02:57.197453 12562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:02:57.232965 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12440
2026-08-08T07:02:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1540 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12582](info): set up 2 sockets
Aug 08 07:02:57 dist-test-slave-nfb5 krb5kdc[12582](info): commencing operation
krb5kdc: starting...
W20260808 07:02:58.451735 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:02:58 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:02:58.574445 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:58.655699 12606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:58.655932 12606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:58.655972 12606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:58.657101 12606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:58.657166 12606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:58.657191 12606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:58.657223 12606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:58.657246 12606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:58.658706 12606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:58.659013 12606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:58.659237 12606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:58.664136 12612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:58.664113 12616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:58.664109 12613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:58.664386 12606 server_base.cc:1034] running on GCE node
I20260808 07:02:58.664640 12606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:58.664937 12606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:58.666101 12606 hybrid_clock.cc:648] HybridClock initialized: now 1786172578666096 us; error 39 us; skew 500 ppm
Aug 08 07:02:58 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172578, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:58.668632 12606 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:58.669127 12606 webserver.cc:460] Webserver started at http://127.24.181.126:37451/ using document root <none> and password file <none>
I20260808 07:02:58.669354 12606 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:58.669423 12606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:58.669571 12606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:58.670593 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "90a98517f6084ababec0f27e4cb337ad"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "2e2123f020628bd2517255679c077da7"
server_key_iv: "2e7b7a78b4b2bf83d383d57ac62c72ee"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.671026 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "90a98517f6084ababec0f27e4cb337ad"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "2e2123f020628bd2517255679c077da7"
server_key_iv: "2e7b7a78b4b2bf83d383d57ac62c72ee"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.672349 12606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:58.673058 12625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:58.673237 12606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:58.673338 12606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "90a98517f6084ababec0f27e4cb337ad"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "2e2123f020628bd2517255679c077da7"
server_key_iv: "2e7b7a78b4b2bf83d383d57ac62c72ee"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.673422 12606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:58.691253 12606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:58.693096 12606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:58.693271 12606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:58.697474 12606 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:34431
I20260808 07:02:58.697515 12697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:34431 every 8 connection(s)
I20260808 07:02:58.697896 12606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:58.699102 12698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:58.699398 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12606
I20260808 07:02:58.699545 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:02:58.699793 25301 external_mini_cluster.cc:1428] Setting key 040b09da0a48a1f87b587f4db62d578d
I20260808 07:02:58.702852 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap starting.
I20260808 07:02:58.703707 12698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:58.704053 12698 log.cc:824] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:58.704900 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: No bootstrap required, opened a new log
Aug 08 07:02:58 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:58.707830 12698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:58.708037 12698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:58.708096 12698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Initialized, Role: FOLLOWER
I20260808 07:02:58.708249 12698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:58.708359 12698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:58.708411 12698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:58.708467 12698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:58.709295 12698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:58.709396 12698 leader_election.cc:302] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90a98517f6084ababec0f27e4cb337ad; no voters:
I20260808 07:02:58.709594 12698 leader_election.cc:288] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:58.709630 12703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:58.709787 12703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 LEADER]: Becoming Leader. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Running, Role: LEADER
I20260808 07:02:58.709893 12698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:58.709966 12703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:58.710412 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90a98517f6084ababec0f27e4cb337ad. Latest consensus state: current_term: 1 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:02:58.710528 12707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:02:58.710546 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:58.710662 12707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:58.710944 12714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:58.711771 12714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:58.712389 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:58.700361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.24.181.126:34431)
0808 07:02:58.700532 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:58.700540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:58.700564 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:02:58.701010 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:58.701017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:58.701062 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:58.701173 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:58.702064 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.703021 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.704228 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.704632 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.704734 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:58.707530 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:58.707535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:58.707535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:58.707548 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:58.709842 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:58.710329 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:58.710349 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:58.710350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:58.710460 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:58.710767 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:58.710768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:58.710768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:58.711157 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:58.711857 (+ 700us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:58.711934 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:02:58.712214 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260808 07:02:58.713661 12714 catalog_manager.cc:1347] Generated new cluster ID: d22e6c6b695540d3b213da4e8f3d1b6b
I20260808 07:02:58.713726 12714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:58.728758 12714 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:58.729512 12714 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:58.735540 12714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Generated new TSK 0
I20260808 07:02:58.735733 12714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:02:58.769163 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:58.856078 12731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:58.856299 12731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:58.856324 12731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:58.857472 12731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:58.857513 12731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:58.857538 12731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:02:58.859129 12731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:58.859372 12731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:58.859622 12731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:58.864838 12738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:58.864866 12737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:58.864873 12743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:58.865238 12731 server_base.cc:1034] running on GCE node
I20260808 07:02:58.865455 12731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:58.865725 12731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:58.866907 12731 hybrid_clock.cc:648] HybridClock initialized: now 1786172578866889 us; error 28 us; skew 500 ppm
Aug 08 07:02:58 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172578, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:58.869629 12731 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:02:58.870124 12731 webserver.cc:460] Webserver started at http://127.24.181.65:40231/ using document root <none> and password file <none>
I20260808 07:02:58.870378 12731 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:58.870455 12731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:58.870615 12731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:58.871831 12731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "4245b35c807249edb2caab8580bc2cf3"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "8b44e4347bd2dc84d7db1f8239c9d65e"
server_key_iv: "684fa748b59d0ebe142de47e2679f193"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.872224 12731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "4245b35c807249edb2caab8580bc2cf3"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "8b44e4347bd2dc84d7db1f8239c9d65e"
server_key_iv: "684fa748b59d0ebe142de47e2679f193"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.873724 12731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:58.874527 12756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:58.874698 12731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:58.874804 12731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "4245b35c807249edb2caab8580bc2cf3"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "8b44e4347bd2dc84d7db1f8239c9d65e"
server_key_iv: "684fa748b59d0ebe142de47e2679f193"
server_key_version: "encryptionkey@0"
I20260808 07:02:58.874892 12731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:58.892858 12731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:58.895282 12731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:58.895458 12731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:58.895702 12731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:58.896037 12731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:58.896100 12731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:58.896143 12731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:58.896171 12731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:58.902026 12731 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:37829
I20260808 07:02:58.902083 12909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:37829 every 8 connection(s)
I20260808 07:02:58.902419 12731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:58.904469 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12731
I20260808 07:02:58.904560 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:58.904762 25301 external_mini_cluster.cc:1428] Setting key a16ece1e51f8f6aefdf135a813e3fc74
Aug 08 07:02:58 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172578, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:58.912556 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:58.903162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:36741 (local address 127.24.181.126:34431)
0808 07:02:58.903255 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:58.903257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:58.903390 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:02:58.903863 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:58.903864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:58.903875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:58.903918 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:58.904910 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.905627 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.906555 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.906915 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.907006 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:58.909374 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:58.909375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:58.909376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:58.909384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:58.911141 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:58.911544 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:58.911546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:58.911546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:58.911614 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:58.911889 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:58.911890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:58.911891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:58.912037 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:58.912121 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:58.912345 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:02:58.912442 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:02:58.912567 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:58.903112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:36741)
0808 07:02:58.903265 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:58.903300 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:58.903767 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:58.903968 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:58.903971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:58.904542 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:58.904762 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:58.904766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.905715 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.905716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:58.906468 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:58.906470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:58.907020 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:58.907022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:58.907179 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:58.907656 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:58.907666 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:58.909250 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:58.911237 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:58.911243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:58.911245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:58.911467 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:58.911691 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:58.911692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:58.911692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:58.911788 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:58.912108 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:58.912111 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:58.912219 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 07:02:58.912297 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:02:58.912415 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260808 07:02:58.913168 12912 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:02:58.913278 12912 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:58.913472 12912 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:02:58.913991 12647 ts_manager.cc:194] Registered new tserver with Master: 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:02:58.914657 12647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:36741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:02:58.931986 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:59.012935 12924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:59.013137 12924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:59.013162 12924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:59.014318 12924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:59.014372 12924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:59.014424 12924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:02:59.015890 12924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:59.016137 12924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:59.016384 12924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:59.021385 12933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.021436 12931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.021416 12930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.021613 12924 server_base.cc:1034] running on GCE node
I20260808 07:02:59.021806 12924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:59.022054 12924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:59.023216 12924 hybrid_clock.cc:648] HybridClock initialized: now 1786172579023208 us; error 35 us; skew 500 ppm
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:59.025715 12924 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:02:59.026141 12924 webserver.cc:460] Webserver started at http://127.24.181.66:41361/ using document root <none> and password file <none>
I20260808 07:02:59.026463 12924 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:59.026532 12924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:59.026672 12924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:59.027777 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "0c96c55acf06825e70b9e6b88cf5913c"
server_key_iv: "aae4bcc1c3afb49daac47bffb6d309bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.028151 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "0c96c55acf06825e70b9e6b88cf5913c"
server_key_iv: "aae4bcc1c3afb49daac47bffb6d309bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.029452 12924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:59.030176 12942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.030356 12924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:59.030455 12924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "0c96c55acf06825e70b9e6b88cf5913c"
server_key_iv: "aae4bcc1c3afb49daac47bffb6d309bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.030543 12924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:59.050482 12924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:59.052848 12924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:59.053016 12924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:59.053265 12924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:59.053578 12924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:59.053632 12924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.053668 12924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:59.053697 12924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.059753 12924 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:43901
I20260808 07:02:59.059787 13100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:43901 every 8 connection(s)
I20260808 07:02:59.060101 12924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:59.067044 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 12924
I20260808 07:02:59.067209 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:59.067437 25301 external_mini_cluster.cc:1428] Setting key 26bcef70e52ca8745a93cc92a6dfbb16
I20260808 07:02:59.069732 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.060764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:48673 (local address 127.24.181.126:34431)
0808 07:02:59.060835 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.060838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.061019 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:02:59.061558 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.061559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.061571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.061631 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.062543 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.063121 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.064107 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.064315 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.064406 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.066577 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.066578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.066579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.066587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.068338 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.068730 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.068731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.068732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.068797 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.069038 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.069039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.069040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.069185 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.069254 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.069488 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.069588 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:02:59.069796 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.060717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:48673)
0808 07:02:59.060888 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.060933 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.061449 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.061696 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.061700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.062258 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.062443 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.062448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.063214 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.063215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.064002 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.064004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.064419 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.064420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.064549 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.065040 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.065049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.066481 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.068421 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.068428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.068430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.068654 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.068870 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.068870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.068871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.068972 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.069250 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.069254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.069366 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.069445 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.069559 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260808 07:02:59.070267 13101 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:02:59.070412 13101 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:59.070611 13101 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:02:59.071112 12647 ts_manager.cc:194] Registered new tserver with Master: 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
I20260808 07:02:59.071612 12647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:48673
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:02:59.094084 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:59.177810 13109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:59.178030 13109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:59.178056 13109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:59.179412 13109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:59.179461 13109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:59.179513 13109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:02:59.180974 13109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:59.181208 13109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:59.181447 13109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:59.186604 13121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.186630 13117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.186612 13116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.186949 13109 server_base.cc:1034] running on GCE node
I20260808 07:02:59.187134 13109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:59.187323 13109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:59.188470 13109 hybrid_clock.cc:648] HybridClock initialized: now 1786172579188456 us; error 30 us; skew 500 ppm
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:59.191087 13109 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:02:59.191552 13109 webserver.cc:460] Webserver started at http://127.24.181.67:35201/ using document root <none> and password file <none>
I20260808 07:02:59.191794 13109 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:59.191865 13109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:59.192023 13109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:59.193069 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "ceeafc8e7c047c4835b087dc533ea883"
server_key_iv: "4daa86eee855619575c559ebc118184e"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.193471 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "ceeafc8e7c047c4835b087dc533ea883"
server_key_iv: "4daa86eee855619575c559ebc118184e"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.194795 13109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:02:59.195598 13129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.195770 13109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:59.195878 13109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "ceeafc8e7c047c4835b087dc533ea883"
server_key_iv: "4daa86eee855619575c559ebc118184e"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.195963 13109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:59.228031 13109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:59.230508 13109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:59.230677 13109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:59.230970 13109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:59.231315 13109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:59.231376 13109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.231420 13109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:59.231449 13109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.237725 13109 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:42781
I20260808 07:02:59.237802 13290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:42781 every 8 connection(s)
I20260808 07:02:59.238152 13109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:59.239449 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 13109
I20260808 07:02:59.239539 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:59.239730 25301 external_mini_cluster.cc:1428] Setting key e4c0d6a4562e56621f9aadf6791482a9
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:59.247572 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.238902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:60069 (local address 127.24.181.126:34431)
0808 07:02:59.238956 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.238958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.239203 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:02:59.239855 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.239857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.239870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.239924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.241070 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.241571 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.242575 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.242778 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.242872 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.244929 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.244930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.244931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.244939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.246111 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.246528 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.246529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.246529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.246593 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.246871 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.246872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.246872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.247035 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.247115 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.247399 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.247481 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:02:59.247761 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.238861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:60069)
0808 07:02:59.239080 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.239117 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.239741 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.239995 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.239999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.240765 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.240959 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.240964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.241676 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.241677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.242483 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.242485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.242886 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.242887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.243034 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.243484 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.243492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.244833 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.246202 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.246210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.246213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.246448 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.246674 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.246674 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.246675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.246798 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.247105 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.247113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.247252 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.247366 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.247525 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
I20260808 07:02:59.248278 13291 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:02:59.248390 13291 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:59.248543 13291 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:02:59.248893 12647 ts_manager.cc:194] Registered new tserver with Master: 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781)
I20260808 07:02:59.249260 12647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:60069
I20260808 07:02:59.251511 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:59.258104 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.252813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.24.181.126:34431)
0808 07:02:59.252870 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.252872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.252985 (+ 113us) server_negotiation.cc:407] Connection header received
0808 07:02:59.253026 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.253027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.253034 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.253079 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.253567 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.254004 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.254663 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.254838 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.254886 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.256064 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.256065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.256065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.256069 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.257190 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.257494 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.257494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.257494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.257542 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.257679 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.257680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.257680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.257805 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.257887 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.257972 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.258024 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:02:59.259793 12647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:59.261389 12647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:59.267983 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.263819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43901 (local address 127.0.0.1:52740)
0808 07:02:59.264129 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.264139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.264177 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.264505 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.264509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.264511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:59.264575 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.264577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.265625 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.265626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.266788 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.266791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.267653 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.267654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.267732 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.267745 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.267854 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.267909 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":99,"threads_started":1}
I20260808 07:02:59.267997 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.264080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.24.181.66:43901)
0808 07:02:59.264292 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.264297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.264307 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:59.264346 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.264348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.264361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.264487 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:59.264667 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.265521 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.266934 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.267568 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.267654 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.267713 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.267874 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.267915 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260808 07:02:59.268260 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.263763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42781 (local address 127.0.0.1:57974)
0808 07:02:59.263923 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.263961 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.264021 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.264595 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.264597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.264600 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:59.264658 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.264660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.266031 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.266032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.267103 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.267105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.268038 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.268039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.268105 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.268108 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.268139 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.268181 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260808 07:02:59.268371 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.263819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.0.0.1:35264)
0808 07:02:59.264250 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.264259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.264294 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.264895 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.264896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.264898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:59.264935 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.264937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.266146 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.266147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.267203 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.267207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.268101 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.268101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.268159 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.268161 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.268215 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.268245 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":13,"thread_start_us":100,"threads_started":1}
I20260808 07:02:59.268369 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.264324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.24.181.65:37829)
0808 07:02:59.264653 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.264657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.264666 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:59.264709 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.264711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.264726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.264874 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:59.265010 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.266033 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.267286 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.267955 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.268018 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.268071 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.268234 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.268278 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":112,"threads_started":1}
I20260808 07:02:59.268606 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.263843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.24.181.67:42781)
0808 07:02:59.264418 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.264422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.264433 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:59.264473 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.264474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.264489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.264735 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:59.264795 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.265914 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.267196 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.267925 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.268398 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.268472 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.268500 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.268539 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":79,"threads_started":1}
I20260808 07:02:59.269224 13191 tablet_service.cc:1467] Processing CreateTablet for tablet b7bf439313224514ba86553014ff819d (DEFAULT_TABLE table=test-table [id=c5236bb1f18745a7a15990fd8bc87493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:59.269223 12818 tablet_service.cc:1467] Processing CreateTablet for tablet b7bf439313224514ba86553014ff819d (DEFAULT_TABLE table=test-table [id=c5236bb1f18745a7a15990fd8bc87493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:59.269584 13191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bf439313224514ba86553014ff819d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:59.269589 12818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bf439313224514ba86553014ff819d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:59.269758 13011 tablet_service.cc:1467] Processing CreateTablet for tablet b7bf439313224514ba86553014ff819d (DEFAULT_TABLE table=test-table [id=c5236bb1f18745a7a15990fd8bc87493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:59.270053 13011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bf439313224514ba86553014ff819d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:59.272361 13315 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Bootstrap starting.
I20260808 07:02:59.273033 13316 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Bootstrap starting.
I20260808 07:02:59.273083 13315 tablet_bootstrap.cc:654] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:59.273718 13317 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Bootstrap starting.
I20260808 07:02:59.273769 13316 tablet_bootstrap.cc:654] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:59.273751 13315 log.cc:824] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.274287 13316 log.cc:824] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.274542 13317 tablet_bootstrap.cc:654] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:59.274639 13315 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: No bootstrap required, opened a new log
I20260808 07:02:59.274714 13315 ts_tablet_manager.cc:1397] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:59.275127 13316 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: No bootstrap required, opened a new log
I20260808 07:02:59.275207 13316 ts_tablet_manager.cc:1397] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:59.275242 13317 log.cc:824] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.276049 13317 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: No bootstrap required, opened a new log
I20260808 07:02:59.276119 13317 ts_tablet_manager.cc:1397] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:59.276741 13315 raft_consensus.cc:348] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.276930 13315 raft_consensus.cc:374] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:59.276978 13315 raft_consensus.cc:729] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0192258a7d7d48e5a55ccc3f3a156edc, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.277107 13315 consensus_queue.cc:260] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.277398 13315 ts_tablet_manager.cc:1428] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:02:59.277428 13291 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
I20260808 07:02:59.277361 13316 raft_consensus.cc:348] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.277812 13316 raft_consensus.cc:374] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:59.277855 13316 raft_consensus.cc:729] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4245b35c807249edb2caab8580bc2cf3, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.277961 13316 consensus_queue.cc:260] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.278223 13316 ts_tablet_manager.cc:1428] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:02:59.278504 12912 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
I20260808 07:02:59.278635 13317 raft_consensus.cc:348] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.278784 13317 raft_consensus.cc:374] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:59.278836 13317 raft_consensus.cc:729] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be22706cc4a4fa9b8c8597ef794e1c0, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.278968 13317 consensus_queue.cc:260] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.279277 13317 ts_tablet_manager.cc:1428] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 07:02:59.279330 13101 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
W20260808 07:02:59.310667 13103 tablet.cc:2369] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:59.403265 12915 tablet.cc:2369] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:59.461551 13322 raft_consensus.cc:482] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:59.461688 13322 raft_consensus.cc:504] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.462181 13322 leader_election.cc:288] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901), 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
I20260808 07:02:59.465152 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.462477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:56907 (local address 127.24.181.66:43901)
0808 07:02:59.462540 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.462542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.462553 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:59.462594 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.462596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.462606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.462689 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:59.462953 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.463446 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464418 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.464790 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464878 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.464887 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.465046 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.465087 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:02:59.465152 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.462397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43901 (local address 127.24.181.67:56907)
0808 07:02:59.462478 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.462491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.462539 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.462771 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.462772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.462774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:59.462839 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.462841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.463555 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.463556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.464295 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.464297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.464878 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.464957 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.464960 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.465035 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.465078 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 07:02:59.465233 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.462477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:56517 (local address 127.24.181.65:37829)
0808 07:02:59.462526 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.462529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.462694 (+ 165us) server_negotiation.cc:407] Connection header received
0808 07:02:59.462815 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.462816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.462827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.462896 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:59.463091 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.463583 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464513 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.464865 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464945 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.464952 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.465105 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.465138 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:02:59.465287 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.462407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.24.181.67:56517)
0808 07:02:59.462613 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.462624 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.462686 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.462937 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.462938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.462940 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:59.463013 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.463014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.463674 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.463675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.464425 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.464427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.464946 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.464947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.465021 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.465024 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.465097 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.465137 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":15,"thread_start_us":128,"threads_started":1}
I20260808 07:02:59.465619 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bf439313224514ba86553014ff819d" candidate_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" is_pre_election: true
I20260808 07:02:59.465747 12846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bf439313224514ba86553014ff819d" candidate_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4245b35c807249edb2caab8580bc2cf3" is_pre_election: true
I20260808 07:02:59.465797 13040 raft_consensus.cc:2454] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0192258a7d7d48e5a55ccc3f3a156edc in term 0.
I20260808 07:02:59.465905 12846 raft_consensus.cc:2454] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0192258a7d7d48e5a55ccc3f3a156edc in term 0.
I20260808 07:02:59.466076 13136 leader_election.cc:302] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0192258a7d7d48e5a55ccc3f3a156edc, 4245b35c807249edb2caab8580bc2cf3; no voters:
I20260808 07:02:59.466244 13322 raft_consensus.cc:2781] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:59.466312 13322 raft_consensus.cc:482] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:59.466328 13322 raft_consensus.cc:3037] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:59.467199 13322 raft_consensus.cc:504] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.467358 13322 leader_election.cc:288] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [CANDIDATE]: Term 1 election: Requested vote from peers 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901), 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
I20260808 07:02:59.467552 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bf439313224514ba86553014ff819d" candidate_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
I20260808 07:02:59.467576 12846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bf439313224514ba86553014ff819d" candidate_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4245b35c807249edb2caab8580bc2cf3"
I20260808 07:02:59.467639 13040 raft_consensus.cc:3037] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:59.467645 12846 raft_consensus.cc:3037] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:59.468564 13040 raft_consensus.cc:2454] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0192258a7d7d48e5a55ccc3f3a156edc in term 1.
I20260808 07:02:59.468564 12846 raft_consensus.cc:2454] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0192258a7d7d48e5a55ccc3f3a156edc in term 1.
I20260808 07:02:59.468744 13136 leader_election.cc:302] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0192258a7d7d48e5a55ccc3f3a156edc, 4245b35c807249edb2caab8580bc2cf3; no voters:
I20260808 07:02:59.468853 13322 raft_consensus.cc:2781] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:59.469045 13322 raft_consensus.cc:686] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 LEADER]: Becoming Leader. State: Replica: 0192258a7d7d48e5a55ccc3f3a156edc, State: Running, Role: LEADER
I20260808 07:02:59.469175 13322 consensus_queue.cc:237] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.470077 12647 catalog_manager.cc:5495] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67). New cstate: current_term: 1 leader_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } health_report { overall_health: UNKNOWN } } }
W20260808 07:02:59.488744 13292 tablet.cc:2369] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:59.495050 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.492505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.24.181.67:42781)
0808 07:02:59.492571 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.492573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.492589 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:02:59.492640 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.492641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.492665 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.492754 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:59.492933 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.493427 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.494134 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.494349 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.494409 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.494590 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:59.494707 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:59.494774 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.494848 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.494866 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:02:59.497076 13040 raft_consensus.cc:1261] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Refusing update from remote peer 0192258a7d7d48e5a55ccc3f3a156edc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:59.497076 12846 raft_consensus.cc:1261] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Refusing update from remote peer 0192258a7d7d48e5a55ccc3f3a156edc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:59.497493 13327 consensus_queue.cc:1035] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:59.497678 13322 consensus_queue.cc:1035] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:59.500268 13331 mvcc.cc:205] Tried to move back new op lower bound from 7316162885616664576 to 7316162885506457600. Current Snapshot: MvccSnapshot[applied={T|T < 7316162885616664576}]
I20260808 07:02:59.502849 12646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:59.502981 12646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:59.503772 12646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Sending DeleteTablet for 3 replicas of tablet b7bf439313224514ba86553014ff819d
I20260808 07:02:59.504122 13191 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bf439313224514ba86553014ff819d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:59 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:57974
I20260808 07:02:59.504153 13011 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bf439313224514ba86553014ff819d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:59 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:52740
I20260808 07:02:59.504354 13350 tablet_replica.cc:330] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: stopping tablet replica
I20260808 07:02:59.504375 12818 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bf439313224514ba86553014ff819d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:59 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:35264
I20260808 07:02:59.504460 13350 raft_consensus.cc:2229] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:59.504572 13350 raft_consensus.cc:2258] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:59.504604 13352 tablet_replica.cc:330] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: stopping tablet replica
I20260808 07:02:59.504642 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12731
I20260808 07:02:59.504758 13352 raft_consensus.cc:2229] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:59.504876 13352 raft_consensus.cc:2258] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:59.521301 13351 tablet_replica.cc:330] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: stopping tablet replica
I20260808 07:02:59.521478 13351 raft_consensus.cc:2229] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:59.521766 13351 raft_consensus.cc:2258] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:59.522338 13351 ts_tablet_manager.cc:1905] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:59.524242 13351 ts_tablet_manager.cc:1918] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:59.524361 13351 log.cc:1196] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/b7bf439313224514ba86553014ff819d
I20260808 07:02:59.524643 13351 ts_tablet_manager.cc:1939] T b7bf439313224514ba86553014ff819d P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting consensus metadata
I20260808 07:02:59.525084 12628 catalog_manager.cc:4841] TS 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): tablet b7bf439313224514ba86553014ff819d (table test-table [id=c5236bb1f18745a7a15990fd8bc87493]) successfully deleted
W20260808 07:02:59.527392 12629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:37829 (error 108)
I20260808 07:02:59.527688 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12924
I20260808 07:02:59.527967 13350 ts_tablet_manager.cc:1905] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:59.532997 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.532827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.0.0.1:35272)
0808 07:02:59.532899 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.532953 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 07:02:59.533185 12629 catalog_manager.cc:4584] TS 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7bf439313224514ba86553014ff819d: Network error: Client connection negotiation failed: client connection to 127.24.181.65:37829: connect: Connection refused (error 111)
I20260808 07:02:59.537050 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.536924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43901 (local address 127.0.0.1:52746)
0808 07:02:59.536981 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.537014 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.66:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:02:59.537053 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 13109
I20260808 07:02:59.554327 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 12606
I20260808 07:02:59.567339 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--webserver_interface=127.24.181.126
--webserver_port=37451
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:59.649056 13356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:59.649262 13356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:59.649284 13356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:59.650502 13356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:59.650547 13356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:59.650561 13356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:59.650574 13356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:59.650633 13356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:59.652132 13356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=37451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:59.652407 13356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:59.652635 13356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:59.661954 13367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.661960 13370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.662031 13356 server_base.cc:1034] running on GCE node
W20260808 07:02:59.662142 13365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.662482 13356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:59.662843 13356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:59.664013 13356 hybrid_clock.cc:648] HybridClock initialized: now 1786172579664003 us; error 33 us; skew 500 ppm
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:59.666631 13356 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:02:59.667165 13356 webserver.cc:460] Webserver started at http://127.24.181.126:37451/ using document root <none> and password file <none>
I20260808 07:02:59.667438 13356 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:59.667522 13356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:59.668951 13356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:59.669617 13377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.669808 13356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:59.669921 13356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "90a98517f6084ababec0f27e4cb337ad"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "2e2123f020628bd2517255679c077da7"
server_key_iv: "2e7b7a78b4b2bf83d383d57ac62c72ee"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.670363 13356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:59.682440 13356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:59.684329 13356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:59.684495 13356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:59.688891 13356 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:34431
I20260808 07:02:59.688926 13460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:34431 every 8 connection(s)
I20260808 07:02:59.689342 13356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:59.691026 13461 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:02:59.691646 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap starting.
I20260808 07:02:59.692401 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 13356
I20260808 07:02:59.692739 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:37829
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=40231
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:59.701719 13461 log.cc:824] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.703750 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:59.704082 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap complete.
I20260808 07:02:59.706462 13461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:59.706682 13461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.706897 13461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:59.707031 13461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:59.707096 13461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:59.707166 13461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:02:59.708171 13461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:59.708293 13461 leader_election.cc:302] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90a98517f6084ababec0f27e4cb337ad; no voters:
I20260808 07:02:59.708496 13461 leader_election.cc:288] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:02:59.708534 13465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:02:59.708649 13465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 LEADER]: Becoming Leader. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Running, Role: LEADER
I20260808 07:02:59.708779 13461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:59.708814 13465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:02:59.709232 13466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:02:59.709338 13466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:59.709694 13467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90a98517f6084ababec0f27e4cb337ad. Latest consensus state: current_term: 2 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:02:59.709764 13467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:59.709908 13473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:59.710811 13473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:59.710943 13473 catalog_manager.cc:1259] Loaded cluster ID: d22e6c6b695540d3b213da4e8f3d1b6b
I20260808 07:02:59.710973 13473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:59.711254 13473 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 07:02:59.714078 13473 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 07:02:59.715970 13473 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:59.716192 13473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Loaded TSK: 0
I20260808 07:02:59.716434 13473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:02:59.786377 13463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:59.786579 13463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:59.786602 13463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:59.787813 13463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:59.787860 13463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:59.787889 13463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:02:59.789361 13463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:37829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=40231
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:59.789600 13463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:59.789856 13463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:59.796200 13496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.796416 13494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.796252 13463 server_base.cc:1034] running on GCE node
W20260808 07:02:59.796634 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.796921 13463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:59.797163 13463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:59.798319 13463 hybrid_clock.cc:648] HybridClock initialized: now 1786172579798305 us; error 30 us; skew 500 ppm
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:59.800863 13463 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:02:59.801304 13463 webserver.cc:460] Webserver started at http://127.24.181.65:40231/ using document root <none> and password file <none>
I20260808 07:02:59.801559 13463 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:59.801654 13463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:59.803059 13463 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:59.803819 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.804009 13463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:59.804109 13463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "4245b35c807249edb2caab8580bc2cf3"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "8b44e4347bd2dc84d7db1f8239c9d65e"
server_key_iv: "684fa748b59d0ebe142de47e2679f193"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.804461 13463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:59.812263 13463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:59.814512 13463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:59.814672 13463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:59.814898 13463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:59.815375 13517 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:02:59.817343 13463 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:02:59.817402 13463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:02:59.817444 13463 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:02:59.818080 13463 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:02:59.818138 13463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:02:59.818199 13517 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Bootstrap starting.
I20260808 07:02:59.825760 13463 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:37829
I20260808 07:02:59.825807 13658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:37829 every 8 connection(s)
I20260808 07:02:59.826203 13463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:59.828857 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 13463
I20260808 07:02:59.829173 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:43901
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=41361
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:59.830988 13517 log.cc:824] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.833729 13517 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:59.834051 13517 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Bootstrap complete.
I20260808 07:02:59.835498 13517 ts_tablet_manager.cc:1397] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.010s
I20260808 07:02:59.836989 13517 raft_consensus.cc:348] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.837395 13517 raft_consensus.cc:729] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4245b35c807249edb2caab8580bc2cf3, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.837620 13517 consensus_queue.cc:260] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:59.837996 13517 ts_tablet_manager.cc:1428] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:02:59.843538 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.826897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:60951)
0808 07:02:59.827114 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.827156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.827595 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.831590 (+ 3995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.831595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.832017 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.832443 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.832451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.833639 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.833640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.834581 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.834584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.835467 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.835469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.835628 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.836305 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.836326 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.838457 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.841400 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.841408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.841411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.841743 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.842086 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.842087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.842088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.842233 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.842787 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.842794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.842978 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.843118 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.843320 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":123,"threads_started":1}
I20260808 07:02:59.843906 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.827026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:60951 (local address 127.24.181.126:34431)
0808 07:02:59.831084 (+ 4058us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.831094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.831142 (+ 48us) server_negotiation.cc:407] Connection header received
0808 07:02:59.831290 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.831297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.831368 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.832556 (+ 1188us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.832628 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.833515 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.834680 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.835353 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.838620 (+ 3267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.838712 (+ 92us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.838715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.838716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.838726 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.841278 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.841851 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.841855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.841856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.841986 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.842316 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.842325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.842326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.842677 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.842769 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.843257 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.843437 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3990,"thread_start_us":88,"threads_started":1}
I20260808 07:02:59.844563 13660 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:02:59.844662 13660 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:59.844851 13660 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:02:59.845639 13404 ts_manager.cc:194] Registered new tserver with Master: 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
I20260808 07:02:59.846872 13404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:60951
I20260808 07:02:59.847885 13660 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:02:59.858248 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.846291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.0.0.1:35276)
0808 07:02:59.847185 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.847216 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.847288 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.847568 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.847573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.847911 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.847998 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.848001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.849498 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.849499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.850485 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.850489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.851239 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.851240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.851341 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.851983 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.851989 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.854535 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.856953 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.856959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.856961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.857211 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.857455 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.857456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.857456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.857521 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.857879 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.857885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.857935 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.858007 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.858142 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":109,"threads_started":1}
I20260808 07:02:59.858959 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.846555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.24.181.65:37829)
0808 07:02:59.847086 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.847091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.847381 (+ 290us) server_negotiation.cc:407] Connection header received
0808 07:02:59.847420 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.847422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.847438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.847494 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.848087 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.849403 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.850610 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.851113 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.851179 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.854637 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.854646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.854647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.854654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.856832 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.857311 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.857314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.857315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.857380 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.857589 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.857590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.857590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.857778 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.857852 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.858690 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.858802 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":79,"threads_started":1}
I20260808 07:02:59.859493 13576 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bf439313224514ba86553014ff819d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:59 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:35276
I20260808 07:02:59.860164 13671 tablet_replica.cc:330] stopping tablet replica
I20260808 07:02:59.860268 13671 raft_consensus.cc:2229] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:59.860345 13671 raft_consensus.cc:2258] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:59.860713 13671 ts_tablet_manager.cc:1905] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:59.863036 13671 ts_tablet_manager.cc:1918] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:59.863118 13671 log.cc:1196] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/b7bf439313224514ba86553014ff819d
I20260808 07:02:59.863435 13671 ts_tablet_manager.cc:1939] T b7bf439313224514ba86553014ff819d P 4245b35c807249edb2caab8580bc2cf3: Deleting consensus metadata
I20260808 07:02:59.863978 13382 catalog_manager.cc:4841] TS 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829): tablet b7bf439313224514ba86553014ff819d (table test-table [id=c5236bb1f18745a7a15990fd8bc87493]) successfully deleted
W20260808 07:02:59.924116 13666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:59.924311 13666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:59.924336 13666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:59.925393 13666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:59.925428 13666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:59.925453 13666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:02:59.926805 13666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:43901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=41361
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:02:59.927065 13666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:59.927248 13666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:59.932266 13677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.932274 13678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:59.932274 13680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:59.932638 13666 server_base.cc:1034] running on GCE node
I20260808 07:02:59.932960 13666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:59.933159 13666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:59.934303 13666 hybrid_clock.cc:648] HybridClock initialized: now 1786172579934291 us; error 29 us; skew 500 ppm
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:59.936887 13666 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:02:59.937335 13666 webserver.cc:460] Webserver started at http://127.24.181.66:41361/ using document root <none> and password file <none>
I20260808 07:02:59.937561 13666 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:59.937646 13666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:59.939029 13666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:59.939735 13689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:59.939908 13666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:59.940008 13666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "0c96c55acf06825e70b9e6b88cf5913c"
server_key_iv: "aae4bcc1c3afb49daac47bffb6d309bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:59.940363 13666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:59.959280 13666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:59.961663 13666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:59.961833 13666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:59.962075 13666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:59.962543 13703 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:02:59.963977 13666 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:02:59.964041 13666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:02:59.964084 13666 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:02:59.964748 13666 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:02:59.964807 13666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:02:59.964860 13703 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Bootstrap starting.
I20260808 07:02:59.966559 13703 log.cc:824] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:59.969810 13703 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:59.970134 13703 tablet_bootstrap.cc:492] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Bootstrap complete.
I20260808 07:02:59.970314 13703 ts_tablet_manager.cc:1397] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 07:02:59.971508 13666 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:43901
I20260808 07:02:59.971565 13853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:43901 every 8 connection(s)
I20260808 07:02:59.971863 13666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:59.972122 13703 raft_consensus.cc:348] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.972556 13703 raft_consensus.cc:729] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be22706cc4a4fa9b8c8597ef794e1c0, State: Initialized, Role: FOLLOWER
I20260808 07:02:59.972687 13703 consensus_queue.cc:260] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:02:59.972988 13703 ts_tablet_manager.cc:1428] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent starting tablet: real 0.003s user 0.005s sys 0.001s
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:02:59.978686 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 13666
I20260808 07:02:59.979022 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:42781
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=35201
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:59.981190 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.972548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:44767)
0808 07:02:59.972689 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.972731 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.972886 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.973203 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.973207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.973466 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.973664 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.973669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.974386 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.974387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.975233 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.975236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.975586 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.975587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.975682 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.976119 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.976126 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.977938 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.979692 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.979700 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.979703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.980032 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.980230 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.980232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.980232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.980381 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.980608 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.980615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.980796 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.980925 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.981045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260808 07:02:59.981197 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.972774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:44767 (local address 127.24.181.126:34431)
0808 07:02:59.972821 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.972823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.972833 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:59.972997 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.972999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.973012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.973092 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.973772 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.974284 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.975306 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.975512 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.975568 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.978020 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.978022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.978022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.978028 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.979609 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.980113 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.980113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.980114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.980164 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.980445 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.980446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.980446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.980543 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.980593 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.980972 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.981072 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:59.981760 13855 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:02:59.981844 13855 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:59.982023 13855 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:02:59.982564 13404 ts_manager.cc:194] Registered new tserver with Master: 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
I20260808 07:02:59.983175 13404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:44767
I20260808 07:02:59.983887 13855 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
Aug 08 07:02:59 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.66@KRBTEST.COM
I20260808 07:02:59.991667 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.982845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43901 (local address 127.0.0.1:52754)
0808 07:02:59.982947 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:59.982959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:59.983003 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:59.983299 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:59.983300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:59.983479 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:59.983529 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.983532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.985091 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.985091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.985629 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:59.985631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.986346 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.986347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:59.986395 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.986837 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:59.986841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:59.988342 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:59.990538 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.990539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.990539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.990703 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.990992 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:59.990993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:59.990993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:59.991045 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:59.991385 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:59.991386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:59.991442 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 07:02:59.991512 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:02:59.991602 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 07:02:59.991847 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:59.982980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.24.181.66:43901)
0808 07:02:59.983124 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:59.983129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:59.983138 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:59.983173 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:59.983175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:59.983190 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:59.983620 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:59.983830 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.984989 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.985719 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:59.986259 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:59.986344 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:59.988434 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:59.988444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:59.988444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:59.988453 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:59.990443 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.990789 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.990793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.990793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.990863 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:59.991114 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:59.991115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:59.991115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:59.991300 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:59.991397 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:59.991587 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:02:59.991680 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260808 07:02:59.992332 13762 tablet_service.cc:1514] Processing DeleteTablet for tablet b7bf439313224514ba86553014ff819d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:59 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:52754
I20260808 07:02:59.993021 13862 tablet_replica.cc:330] stopping tablet replica
I20260808 07:02:59.993139 13862 raft_consensus.cc:2229] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:59.993232 13862 raft_consensus.cc:2258] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:59.993546 13862 ts_tablet_manager.cc:1905] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:59.995638 13862 ts_tablet_manager.cc:1918] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:59.995729 13862 log.cc:1196] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/b7bf439313224514ba86553014ff819d
I20260808 07:02:59.995993 13862 ts_tablet_manager.cc:1939] T b7bf439313224514ba86553014ff819d P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting consensus metadata
I20260808 07:02:59.996502 13379 catalog_manager.cc:4841] TS 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): tablet b7bf439313224514ba86553014ff819d (table test-table [id=c5236bb1f18745a7a15990fd8bc87493]) successfully deleted
W20260808 07:03:00.066347 13860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:00.066540 13860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:00.066563 13860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:00.067788 13860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:00.067829 13860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:00.067855 13860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:03:00.069285 13860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:42781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=35201
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:00.069496 13860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:00.069715 13860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:00.074898 13868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.075006 13860 server_base.cc:1034] running on GCE node
W20260808 07:03:00.074901 13869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.074960 13871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.075408 13860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:00.075616 13860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:00.076785 13860 hybrid_clock.cc:648] HybridClock initialized: now 1786172580076776 us; error 40 us; skew 500 ppm
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:00.079417 13860 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:03:00.079875 13860 webserver.cc:460] Webserver started at http://127.24.181.67:35201/ using document root <none> and password file <none>
I20260808 07:03:00.080056 13860 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:00.080106 13860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:00.081396 13860 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:00.081935 13883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.082123 13860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:03:00.082190 13860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "ceeafc8e7c047c4835b087dc533ea883"
server_key_iv: "4daa86eee855619575c559ebc118184e"
server_key_version: "encryptionkey@0"
I20260808 07:03:00.082511 13860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:00.093047 13860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:00.094818 13860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:00.094995 13860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:00.095227 13860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:00.095530 13860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:00.095564 13860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.095585 13860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:00.095625 13860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.101608 13860 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:42781
I20260808 07:03:00.101670 14051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:42781 every 8 connection(s)
I20260808 07:03:00.102005 13860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:00.105258 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 13860
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:00.112556 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.105661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.24.181.126:34431)
0808 07:03:00.105793 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.105795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.105805 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:03:00.105864 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.105865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.105876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.105944 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:00.106556 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.107237 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.108031 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.108262 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.108333 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.109637 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:00.109637 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:00.109638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:00.109643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:00.111298 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111640 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:00.111641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:00.111642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:00.111705 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111922 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:00.111923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:00.111923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:00.112087 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:00.112247 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.112332 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.112419 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260808 07:03:00.112659 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.102732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:59085 (local address 127.24.181.126:34431)
0808 07:03:00.102808 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.102810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.103075 (+ 265us) server_negotiation.cc:407] Connection header received
0808 07:03:00.103570 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.103571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.103581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.103634 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:00.104510 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.104983 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.106276 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.106644 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.106697 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.109299 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:00.109301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:00.109301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:00.109308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:00.111150 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111609 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:00.111611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:00.111611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:00.111671 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111937 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:00.111938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:00.111938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:00.112093 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:00.112167 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.112458 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.112544 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:03:00.112812 14057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.102741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:59085)
0808 07:03:00.102959 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.103010 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:00.103486 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:00.103684 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:00.103687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:00.104248 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:00.104436 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.104441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.105078 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.105079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.106190 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.106193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.106727 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.106728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.106880 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.107443 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:00.107452 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:00.109197 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:00.111226 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111234 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:00.111237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:00.111530 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:00.111740 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:00.111742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:00.111742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:00.111860 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:00.112169 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:00.112177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:00.112314 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 07:03:00.112422 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:03:00.112580 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260808 07:03:00.113271 14052 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34431
I20260808 07:03:00.113380 14052 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:00.113566 14052 heartbeater.cc:511] Master 127.24.181.126:34431 requested a full tablet report, sending...
I20260808 07:03:00.113974 13404 ts_manager.cc:194] Registered new tserver with Master: 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781)
I20260808 07:03:00.114341 13404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:59085
I20260808 07:03:00.124536 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:03:00.127139 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.124906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.24.181.126:34431)
0808 07:03:00.124949 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.124951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.125065 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:03:00.125129 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.125130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.125160 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.125222 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:00.125313 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.125757 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.126448 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.126618 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.126666 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.126797 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:00.126882 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:00.127002 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.127038 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.127054 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:03:00.127442 13404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:03:00.129218 13404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:03:00.131460 13762 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6cc844ea854a698677b20586a77eef (DEFAULT_TABLE table=test-table [id=fcecbec8b82e4b0690416c8ed2869ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:00.131469 13576 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6cc844ea854a698677b20586a77eef (DEFAULT_TABLE table=test-table [id=fcecbec8b82e4b0690416c8ed2869ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:00.131676 13762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6cc844ea854a698677b20586a77eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:00.131703 13576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6cc844ea854a698677b20586a77eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:00.133239 13703 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Bootstrap starting.
I20260808 07:03:00.133502 13517 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Bootstrap starting.
I20260808 07:03:00.133675 13703 tablet_bootstrap.cc:654] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:00.133927 13517 tablet_bootstrap.cc:654] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:00.134546 13517 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: No bootstrap required, opened a new log
I20260808 07:03:00.134572 13703 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: No bootstrap required, opened a new log
I20260808 07:03:00.134617 13517 ts_tablet_manager.cc:1397] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:00.134625 13703 ts_tablet_manager.cc:1397] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:00.134785 13703 raft_consensus.cc:348] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.134883 13703 raft_consensus.cc:374] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:00.134961 13703 raft_consensus.cc:729] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be22706cc4a4fa9b8c8597ef794e1c0, State: Initialized, Role: FOLLOWER
I20260808 07:03:00.135018 13703 consensus_queue.cc:260] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.135015 13517 raft_consensus.cc:348] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.135090 13517 raft_consensus.cc:374] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:00.135115 13703 ts_tablet_manager.cc:1428] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:03:00.135139 13517 raft_consensus.cc:729] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4245b35c807249edb2caab8580bc2cf3, State: Initialized, Role: FOLLOWER
I20260808 07:03:00.135186 13517 consensus_queue.cc:260] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.135286 13517 ts_tablet_manager.cc:1428] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:00.136178 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.131275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42781 (local address 127.0.0.1:57994)
0808 07:03:00.131347 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.131360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:00.131404 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:00.131852 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:00.131854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:00.131855 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:00.131898 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.131899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.133294 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.133295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.134510 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.134514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.135916 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.135918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.135982 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.136002 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 07:03:00.136062 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:03:00.136107 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:03:00.136197 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.131413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.24.181.67:42781)
0808 07:03:00.131677 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.131682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.131693 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:03:00.131732 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.131734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.131751 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.131840 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:00.131969 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.133160 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.134747 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.135837 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.135913 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.135972 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.136079 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.136118 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":118,"threads_started":1}
I20260808 07:03:00.136917 13958 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6cc844ea854a698677b20586a77eef (DEFAULT_TABLE table=test-table [id=fcecbec8b82e4b0690416c8ed2869ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:00.137247 13958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6cc844ea854a698677b20586a77eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:00.140036 14066 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Bootstrap starting.
I20260808 07:03:00.140743 14066 tablet_bootstrap.cc:654] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:00.141265 14066 log.cc:824] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:00.141986 14066 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: No bootstrap required, opened a new log
I20260808 07:03:00.142041 14066 ts_tablet_manager.cc:1397] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:00.143468 14066 raft_consensus.cc:348] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.143594 14066 raft_consensus.cc:374] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:00.143639 14066 raft_consensus.cc:729] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0192258a7d7d48e5a55ccc3f3a156edc, State: Initialized, Role: FOLLOWER
I20260808 07:03:00.143765 14066 consensus_queue.cc:260] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.144037 14066 ts_tablet_manager.cc:1428] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:00.144143 14052 heartbeater.cc:503] Master 127.24.181.126:34431 was elected leader, sending a full tablet report...
W20260808 07:03:00.222438 13856 tablet.cc:2369] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:03:00.292914 13858 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:00.293133 13858 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.293520 13858 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
I20260808 07:03:00.296595 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.293837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:41859 (local address 127.24.181.65:37829)
0808 07:03:00.293914 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.293916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.293945 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:03:00.293982 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.293983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.294006 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.294087 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:00.294263 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.294802 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.295827 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.296258 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.296348 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.296424 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.296484 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.296525 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:03:00.296600 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.293796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:38769 (local address 127.24.181.67:42781)
0808 07:03:00.293847 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.293849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.294037 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:03:00.294091 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.294092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.294103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.294177 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:00.294344 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.294804 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.295819 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.296201 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.296286 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.296296 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.296491 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.296528 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:03:00.296604 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.293739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.24.181.66:41859)
0808 07:03:00.293807 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.293819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:00.293876 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:00.294094 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:00.294096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:00.294098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:00.294180 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.294182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.294894 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.294895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.295725 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.295728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.296346 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.296347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.296421 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.296424 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:03:00.296479 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:03:00.296518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":3}
I20260808 07:03:00.296705 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.293761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42781 (local address 127.24.181.66:38769)
0808 07:03:00.293950 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.293959 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:00.294039 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:00.294181 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:00.294182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:00.294183 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:00.294262 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.294264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.294890 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.294891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.295720 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:00.295722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.296299 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.296300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:00.296389 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.296408 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 07:03:00.296483 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:03:00.296525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":15,"thread_start_us":129,"threads_started":1}
I20260808 07:03:00.296941 13605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6cc844ea854a698677b20586a77eef" candidate_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4245b35c807249edb2caab8580bc2cf3" is_pre_election: true
I20260808 07:03:00.297150 13605 raft_consensus.cc:2454] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be22706cc4a4fa9b8c8597ef794e1c0 in term 0.
I20260808 07:03:00.297161 13985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6cc844ea854a698677b20586a77eef" candidate_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" is_pre_election: true
I20260808 07:03:00.297327 13985 raft_consensus.cc:2454] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be22706cc4a4fa9b8c8597ef794e1c0 in term 0.
I20260808 07:03:00.297405 13697 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3, 9be22706cc4a4fa9b8c8597ef794e1c0; no voters:
I20260808 07:03:00.297536 13858 raft_consensus.cc:2781] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:03:00.297600 13858 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:03:00.297617 13858 raft_consensus.cc:3037] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:00.298295 13858 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.298454 13858 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [CANDIDATE]: Term 1 election: Requested vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829)
I20260808 07:03:00.298601 13605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6cc844ea854a698677b20586a77eef" candidate_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4245b35c807249edb2caab8580bc2cf3"
I20260808 07:03:00.298681 13605 raft_consensus.cc:3037] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:00.299014 13985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6cc844ea854a698677b20586a77eef" candidate_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
I20260808 07:03:00.299108 13985 raft_consensus.cc:3037] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:00.299548 13605 raft_consensus.cc:2454] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be22706cc4a4fa9b8c8597ef794e1c0 in term 1.
I20260808 07:03:00.299747 13697 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3, 9be22706cc4a4fa9b8c8597ef794e1c0; no voters:
I20260808 07:03:00.299837 13858 raft_consensus.cc:2781] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:00.300002 13985 raft_consensus.cc:2454] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be22706cc4a4fa9b8c8597ef794e1c0 in term 1.
I20260808 07:03:00.300021 13858 raft_consensus.cc:686] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 LEADER]: Becoming Leader. State: Replica: 9be22706cc4a4fa9b8c8597ef794e1c0, State: Running, Role: LEADER
I20260808 07:03:00.300096 13858 consensus_queue.cc:237] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.300911 13404 catalog_manager.cc:5495] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66). New cstate: current_term: 1 leader_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } health_report { overall_health: UNKNOWN } } }
I20260808 07:03:00.317719 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.315285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.24.181.66:43901)
0808 07:03:00.315350 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:00.315352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:00.315361 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:03:00.315394 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:00.315395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:00.315411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:00.315491 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:00.315665 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.316158 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.316888 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:00.317096 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:00.317186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:00.317382 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:00.317472 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:00.317552 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:00.317596 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:03:00.317618 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:03:00.319227 13985 raft_consensus.cc:1261] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Refusing update from remote peer 9be22706cc4a4fa9b8c8597ef794e1c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:00.319239 13605 raft_consensus.cc:1261] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Refusing update from remote peer 9be22706cc4a4fa9b8c8597ef794e1c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:00.319607 13858 consensus_queue.cc:1035] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:00.319762 14072 consensus_queue.cc:1035] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:00.321866 14077 mvcc.cc:205] Tried to move back new op lower bound from 7316162888985108480 to 7316162888909864960. Current Snapshot: MvccSnapshot[applied={T|T < 7316162888985108480}]
I20260808 07:03:00.322319 14078 mvcc.cc:205] Tried to move back new op lower bound from 7316162888985108480 to 7316162888909864960. Current Snapshot: MvccSnapshot[applied={T|T < 7316162888985108480}]
I20260808 07:03:00.322309 14079 mvcc.cc:205] Tried to move back new op lower bound from 7316162888985108480 to 7316162888909864960. Current Snapshot: MvccSnapshot[applied={T|T < 7316162888985108480}]
I20260808 07:03:00.324802 13403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:00.324909 13403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:00.325685 13403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Sending DeleteTablet for 3 replicas of tablet 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:00.326040 13762 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6cc844ea854a698677b20586a77eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:00 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:52754
I20260808 07:03:00.326058 13958 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6cc844ea854a698677b20586a77eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:00 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:57994
I20260808 07:03:00.326139 13862 tablet_replica.cc:330] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: stopping tablet replica
I20260808 07:03:00.326190 13862 raft_consensus.cc:2229] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:03:00.326236 13576 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6cc844ea854a698677b20586a77eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:00 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:35276
I20260808 07:03:00.326272 13862 raft_consensus.cc:2258] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:00.326350 13671 tablet_replica.cc:330] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: stopping tablet replica
I20260808 07:03:00.326385 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 13463
I20260808 07:03:00.326412 13671 raft_consensus.cc:2229] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:00.326462 13671 raft_consensus.cc:2258] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:00.326606 13862 ts_tablet_manager.cc:1905] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:00.328336 13862 ts_tablet_manager.cc:1918] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:00.328392 13862 log.cc:1196] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/6d6cc844ea854a698677b20586a77eef
I20260808 07:03:00.328598 13862 ts_tablet_manager.cc:1939] T 6d6cc844ea854a698677b20586a77eef P 9be22706cc4a4fa9b8c8597ef794e1c0: Deleting consensus metadata
I20260808 07:03:00.328913 13379 catalog_manager.cc:4841] TS 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): tablet 6d6cc844ea854a698677b20586a77eef (table test-table [id=fcecbec8b82e4b0690416c8ed2869ba8]) successfully deleted
I20260808 07:03:00.337291 14089 tablet_replica.cc:330] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: stopping tablet replica
I20260808 07:03:00.337459 14089 raft_consensus.cc:2229] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:00.337611 14089 raft_consensus.cc:2258] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:00.339272 14089 ts_tablet_manager.cc:1905] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:00.341331 14089 ts_tablet_manager.cc:1918] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:00.341419 14089 log.cc:1196] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/6d6cc844ea854a698677b20586a77eef
I20260808 07:03:00.341743 14089 ts_tablet_manager.cc:1939] T 6d6cc844ea854a698677b20586a77eef P 0192258a7d7d48e5a55ccc3f3a156edc: Deleting consensus metadata
I20260808 07:03:00.342247 13381 catalog_manager.cc:4841] TS 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): tablet 6d6cc844ea854a698677b20586a77eef (table test-table [id=fcecbec8b82e4b0690416c8ed2869ba8]) successfully deleted
W20260808 07:03:00.344676 13382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:37829 (error 108)
I20260808 07:03:00.345001 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 13666
I20260808 07:03:00.345216 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.345084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:37829 (local address 127.0.0.1:35288)
0808 07:03:00.345142 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.345178 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:03:00.345328 13382 catalog_manager.cc:4584] TS 4245b35c807249edb2caab8580bc2cf3 (127.24.181.65:37829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d6cc844ea854a698677b20586a77eef: Network error: Client connection negotiation failed: client connection to 127.24.181.65:37829: connect: Connection refused (error 111)
I20260808 07:03:00.351727 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 13860
I20260808 07:03:00.364825 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 13356
I20260808 07:03:00.376696 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--webserver_interface=127.24.181.126
--webserver_port=37451
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:00.475051 14091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:00.475302 14091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:00.475342 14091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:00.476632 14091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:03:00.476716 14091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:00.476743 14091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:00.476771 14091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:03:00.476797 14091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:03:00.478430 14091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=37451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:00.478735 14091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:00.479030 14091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:00.483908 14099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.484000 14101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.483973 14098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.484306 14091 server_base.cc:1034] running on GCE node
I20260808 07:03:00.484503 14091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:00.484750 14091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:00.485924 14091 hybrid_clock.cc:648] HybridClock initialized: now 1786172580485918 us; error 36 us; skew 500 ppm
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:00.488559 14091 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:03:00.489019 14091 webserver.cc:460] Webserver started at http://127.24.181.126:37451/ using document root <none> and password file <none>
I20260808 07:03:00.489280 14091 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:00.489329 14091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:00.490685 14091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:00.491451 14109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.491627 14091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.491720 14091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "90a98517f6084ababec0f27e4cb337ad"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "2e2123f020628bd2517255679c077da7"
server_key_iv: "2e7b7a78b4b2bf83d383d57ac62c72ee"
server_key_version: "encryptionkey@0"
I20260808 07:03:00.492054 14091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:00.513011 14091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:00.514809 14091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:00.514971 14091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:00.519035 14091 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:34431
I20260808 07:03:00.519145 14178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:34431 every 8 connection(s)
I20260808 07:03:00.519480 14091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:03:00.521060 14179 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:03:00.521729 14179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap starting.
I20260808 07:03:00.522837 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 14091
I20260808 07:03:00.523218 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:37829
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=40231
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:00.531694 14179 log.cc:824] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:00.533915 14179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:03:00.534252 14179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad: Bootstrap complete.
I20260808 07:03:00.536643 14179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:03:00.536870 14179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Initialized, Role: FOLLOWER
I20260808 07:03:00.537045 14179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:03:00.537106 14179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:03:00.537168 14179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:03:00.537221 14179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:03:00.538237 14179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:03:00.538326 14179 leader_election.cc:302] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90a98517f6084ababec0f27e4cb337ad; no voters:
I20260808 07:03:00.538543 14179 leader_election.cc:288] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:03:00.538578 14188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:03:00.538687 14188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [term 3 LEADER]: Becoming Leader. State: Replica: 90a98517f6084ababec0f27e4cb337ad, State: Running, Role: LEADER
I20260808 07:03:00.538826 14179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:03:00.538838 14188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } }
I20260808 07:03:00.539175 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90a98517f6084ababec0f27e4cb337ad. Latest consensus state: current_term: 3 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:03:00.539248 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:00.539410 14190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "90a98517f6084ababec0f27e4cb337ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90a98517f6084ababec0f27e4cb337ad" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34431 } } }
I20260808 07:03:00.539482 14190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a98517f6084ababec0f27e4cb337ad [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:00.539978 14193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:03:00.540740 14193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:03:00.540874 14193 catalog_manager.cc:1259] Loaded cluster ID: d22e6c6b695540d3b213da4e8f3d1b6b
I20260808 07:03:00.540930 14193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 07:03:00.541464 14193 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc1f99f49ad google::LogMessage::Fail() at ??:0
@ 0x7fc1f99f788a google::LogMessage::SendToLog() at ??:0
@ 0x7fc1f99f44de google::LogMessage::Flush() at ??:0
@ 0x7fc1f99f5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc2019150f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc20191e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc1fb8c20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc1fb8b7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc1fc71b6db start_thread at ??:0
@ 0x7fc1f84f261f clone at ??:0
W20260808 07:03:00.638782 14184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:00.639137 14184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:00.639183 14184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:00.641063 14184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:00.641130 14184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:00.641175 14184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:03:00.643529 14184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:37829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=40231
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:00.643795 14184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:00.644078 14184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:00.649663 14235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.649675 14236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.649801 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.650038 14184 server_base.cc:1034] running on GCE node
I20260808 07:03:00.650347 14184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:00.650583 14184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:00.651746 14184 hybrid_clock.cc:648] HybridClock initialized: now 1786172580651732 us; error 33 us; skew 500 ppm
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:00.654428 14184 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:03:00.654874 14184 webserver.cc:460] Webserver started at http://127.24.181.65:40231/ using document root <none> and password file <none>
I20260808 07:03:00.655128 14184 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:00.655241 14184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:00.656764 14184 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:00.657552 14250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.657773 14184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:00.657881 14184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "4245b35c807249edb2caab8580bc2cf3"
format_stamp: "Formatted at 2026-08-08 07:02:58 on dist-test-slave-nfb5"
server_key: "8b44e4347bd2dc84d7db1f8239c9d65e"
server_key_iv: "684fa748b59d0ebe142de47e2679f193"
server_key_version: "encryptionkey@0"
I20260808 07:03:00.658236 14184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:00.670782 14184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:00.673151 14184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:00.673321 14184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:00.673555 14184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:00.674007 14258 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:03:00.676178 14184 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:03:00.676245 14184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:03:00.676299 14184 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:03:00.676986 14184 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:03:00.677043 14184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:00.677124 14258 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Bootstrap starting.
I20260808 07:03:00.684593 14184 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:37829
I20260808 07:03:00.684749 14419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:37829 every 8 connection(s)
I20260808 07:03:00.685029 14184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:03:00.689640 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 14184
I20260808 07:03:00.689929 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:43901
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=41361
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:00.690572 14258 log.cc:824] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:00.693631 14258 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:03:00.693996 14258 tablet_bootstrap.cc:492] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Bootstrap complete.
I20260808 07:03:00.695634 14258 ts_tablet_manager.cc:1397] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.013s
I20260808 07:03:00.697372 14258 raft_consensus.cc:348] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.697757 14258 raft_consensus.cc:729] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4245b35c807249edb2caab8580bc2cf3, State: Initialized, Role: FOLLOWER
I20260808 07:03:00.697942 14258 consensus_queue.cc:260] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:00.698246 14258 ts_tablet_manager.cc:1428] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260808 07:03:00.776584 14423 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:03:00.685819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:33233)
0808 07:03:00.685999 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.686044 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:00.686510 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:00.776373 (+ 89863us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
W20260808 07:03:00.776446 14426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:00.776696 14426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:00.776746 14426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:00.776871 14420 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:34431 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:34431: Client connection negotiation failed: client connection to 127.24.181.126:34431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:03:00.777267 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.777119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:57409)
0808 07:03:00.777179 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.777239 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:03:00.777653 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.777558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:35551)
0808 07:03:00.777604 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.777632 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:03:00.777779 14420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:03:00.778292 14426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:00.778333 14426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:00.778360 14426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:03:00.779964 14426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:43901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=41361
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:00.780150 14426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:00.780329 14426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:00.785315 14433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.785326 14435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.785398 14432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.785471 14426 server_base.cc:1034] running on GCE node
I20260808 07:03:00.785831 14426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:00.786059 14426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:00.787241 14426 hybrid_clock.cc:648] HybridClock initialized: now 1786172580787222 us; error 28 us; skew 500 ppm
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:00.789870 14426 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:03:00.790323 14426 webserver.cc:460] Webserver started at http://127.24.181.66:41361/ using document root <none> and password file <none>
I20260808 07:03:00.790568 14426 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:00.790614 14426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:00.794545 14426 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 07:03:00.795579 14446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.795809 14426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:00.795907 14426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "9be22706cc4a4fa9b8c8597ef794e1c0"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "0c96c55acf06825e70b9e6b88cf5913c"
server_key_iv: "aae4bcc1c3afb49daac47bffb6d309bf"
server_key_version: "encryptionkey@0"
I20260808 07:03:00.796372 14426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:00.807989 14426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:00.809710 14426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:00.809819 14426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:00.809983 14426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:00.810286 14426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:00.810320 14426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.810343 14426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:00.810392 14426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.816555 14426 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:43901
I20260808 07:03:00.816627 14601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:43901 every 8 connection(s)
I20260808 07:03:00.816977 14426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:03:00.818145 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.817672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:42749)
0808 07:03:00.817857 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.817964 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
W20260808 07:03:00.818535 14603 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:34431 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:34431: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
I20260808 07:03:00.818842 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.818749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:54313)
0808 07:03:00.818796 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.818825 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:03:00.819147 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.819070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:34445)
0808 07:03:00.819110 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.819132 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:03:00.819269 14603 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:03:00.827281 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 14426
I20260808 07:03:00.827572 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:42781
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:34431
--webserver_port=35201
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:00.911134 14607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:00.911327 14607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:00.911351 14607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:00.912410 14607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:00.912446 14607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:00.912472 14607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:03:00.913831 14607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:42781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=35201
--tserver_master_addrs=127.24.181.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:00.914006 14607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:00.914167 14607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:00.919217 14614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:00.919251 14613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.919246 14607 server_base.cc:1034] running on GCE node
W20260808 07:03:00.919220 14617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:00.919718 14607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:00.919975 14607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:00.921159 14607 hybrid_clock.cc:648] HybridClock initialized: now 1786172580921142 us; error 61 us; skew 500 ppm
Aug 08 07:03:00 dist-test-slave-nfb5 krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:00.923702 14607 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:03:00.924156 14607 webserver.cc:460] Webserver started at http://127.24.181.67:35201/ using document root <none> and password file <none>
I20260808 07:03:00.924396 14607 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:00.924469 14607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:00.925797 14607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:00.926388 14625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.926571 14607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:03:00.926672 14607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "0192258a7d7d48e5a55ccc3f3a156edc"
format_stamp: "Formatted at 2026-08-08 07:02:59 on dist-test-slave-nfb5"
server_key: "ceeafc8e7c047c4835b087dc533ea883"
server_key_iv: "4daa86eee855619575c559ebc118184e"
server_key_version: "encryptionkey@0"
I20260808 07:03:00.927047 14607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:00.948823 14607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:00.950673 14607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:00.950837 14607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:00.951102 14607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:00.951432 14607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:00.951493 14607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.951529 14607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:00.951555 14607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:00.957634 14607 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:42781
I20260808 07:03:00.957697 14787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:42781 every 8 connection(s)
I20260808 07:03:00.958015 14607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:00.959097 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.958657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:53383)
0808 07:03:00.958813 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.958898 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260808 07:03:00.959510 14789 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:34431 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:34431: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
I20260808 07:03:00.959856 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.959758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:60163)
0808 07:03:00.959801 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.959838 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:03:00.960146 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:00.960075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:36667)
0808 07:03:00.960111 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:00.960133 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:03:00.960289 14789 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:03:00.963119 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 14607
I20260808 07:03:01.778424 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:01.778145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:54005)
0808 07:03:01.778323 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:01.778374 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":63,"threads_started":1}
I20260808 07:03:01.819810 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:01.819582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:41693)
0808 07:03:01.819707 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:01.819747 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260808 07:03:01.960777 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:01.960574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:57421)
0808 07:03:01.960695 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:01.960725 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260808 07:03:02.004640 14797 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:02.004812 14797 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:02.005198 14797 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
Aug 08 07:03:02 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.67@KRBTEST.COM
Aug 08 07:03:02 dist-test-slave-nfb5 krb5kdc[12582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.66@KRBTEST.COM
I20260808 07:03:02.015722 14802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.005387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42781 (local address 127.24.181.65:60643)
0808 07:03:02.005511 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:02.005520 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:02.005560 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:02.006406 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:02.006409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:02.006803 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:02.007058 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:02.007064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.008192 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.008193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:02.008951 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:02.008953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.009554 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:02.009676 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:02.010335 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:02.010340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:02.011804 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:02.013916 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:02.013923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:02.013925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:02.014215 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:02.014469 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:02.014470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:02.014470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:02.014608 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:02.015151 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:02.015160 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:02.015309 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 07:03:02.015425 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:03:02.015585 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":104,"thread_start_us":78,"threads_started":1}
I20260808 07:03:02.015780 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.005491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:60643 (local address 127.24.181.67:42781)
0808 07:03:02.005659 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:02.005670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:02.005699 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:03:02.006136 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:02.006139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:02.006179 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:02.006331 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:02.007180 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.008054 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009053 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.009457 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009580 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:02.011928 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:02.011932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:02.011933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:02.011946 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:02.013827 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:02.014305 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:02.014308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:02.014309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:02.014396 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:02.014677 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:02.014678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:02.014679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:02.015067 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:02.015159 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:02.015474 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 07:03:02.015633 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260808 07:03:02.015858 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.005367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:43901 (local address 127.24.181.65:55141)
0808 07:03:02.005427 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:02.005439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:02.005496 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:02.006405 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:02.006409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:02.006805 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:02.007058 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:02.007064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.008163 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.008164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:02.008949 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:02.008951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.009560 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:02.009669 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:02.010285 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:02.010293 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:02.012192 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:02.014534 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:02.014535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:02.014535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:02.014743 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:02.015010 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:02.015011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:02.015012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:02.015082 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:02.015526 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:02.015527 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:02.015568 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:03:02.015656 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:03:02.015748 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":6}
I20260808 07:03:02.016220 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.005487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:55141 (local address 127.24.181.66:43901)
0808 07:03:02.005672 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:02.005680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:02.005712 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:03:02.006157 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:02.006159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:02.006195 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:02.006388 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:02.007196 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.008052 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009049 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:02.009462 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:02.009589 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:02.012410 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:02.012414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:02.012415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:02.012426 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:02.014447 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:02.014828 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:02.014830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:02.014830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:02.014906 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:02.015150 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:02.015151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:02.015151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:02.015454 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:02.015531 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:02.015739 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 07:03:02.016144 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
W20260808 07:03:02.017066 14252 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
W20260808 07:03:02.017197 14253 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:02.017289 14253 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3; no voters: 0192258a7d7d48e5a55ccc3f3a156edc, 9be22706cc4a4fa9b8c8597ef794e1c0
I20260808 07:03:02.017433 14797 raft_consensus.cc:2726] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:03:02.779151 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.778896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:40541)
0808 07:03:02.779068 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:02.779105 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260808 07:03:02.820452 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.820238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:48115)
0808 07:03:02.820385 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:02.820415 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260808 07:03:02.961426 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:02.961193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:57115)
0808 07:03:02.961357 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:02.961390 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260808 07:03:03.779850 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:03.779590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:40237)
0808 07:03:03.779755 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:03.779798 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":66,"threads_started":1}
I20260808 07:03:03.821273 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:03.821006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:37599)
0808 07:03:03.821185 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:03.821229 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":64,"threads_started":1}
I20260808 07:03:03.960250 14814 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:03.960356 14814 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:03.960568 14814 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
W20260808 07:03:03.960987 14253 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
W20260808 07:03:03.961094 14252 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:03.961184 14252 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3; no voters: 0192258a7d7d48e5a55ccc3f3a156edc, 9be22706cc4a4fa9b8c8597ef794e1c0
I20260808 07:03:03.961272 14814 raft_consensus.cc:2726] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:03:03.962060 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:03.961861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:48409)
0808 07:03:03.961983 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:03.962020 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260808 07:03:04.780735 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:04.780459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:38141)
0808 07:03:04.780615 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:04.780663 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260808 07:03:04.821916 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:04.821728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:33435)
0808 07:03:04.821845 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:04.821880 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 07:03:04.962796 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:04.962478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:55359)
0808 07:03:04.962666 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:04.962715 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":60,"threads_started":1}
I20260808 07:03:05.781525 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:05.781304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:38881)
0808 07:03:05.781439 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:05.781478 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260808 07:03:05.822649 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:05.822441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:33811)
0808 07:03:05.822576 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:05.822613 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260808 07:03:05.963454 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:05.963253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:33241)
0808 07:03:05.963378 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:05.963413 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260808 07:03:06.782294 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:06.782052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:49223)
0808 07:03:06.782202 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:06.782246 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260808 07:03:06.823346 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:06.823129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:46717)
0808 07:03:06.823268 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:06.823308 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 07:03:06.964180 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:06.963966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:57129)
0808 07:03:06.964103 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:06.964139 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260808 07:03:07.148224 14827 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:07.148325 14827 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:07.148550 14827 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
W20260808 07:03:07.149008 14253 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
W20260808 07:03:07.149065 14252 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:07.149111 14252 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3; no voters: 0192258a7d7d48e5a55ccc3f3a156edc, 9be22706cc4a4fa9b8c8597ef794e1c0
I20260808 07:03:07.149188 14827 raft_consensus.cc:2726] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:03:07.783025 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:07.782754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:38863)
0808 07:03:07.782901 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:07.782979 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260808 07:03:07.824007 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:07.823791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:38567)
0808 07:03:07.823933 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:07.823967 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260808 07:03:07.964828 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:07.964631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:39357)
0808 07:03:07.964753 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:07.964787 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260808 07:03:08.783763 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:08.783550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:40833)
0808 07:03:08.783678 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:08.783715 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260808 07:03:08.824786 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:08.824473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:54269)
0808 07:03:08.824630 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:08.824740 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260808 07:03:08.965497 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:08.965288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:47089)
0808 07:03:08.965417 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:08.965454 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 07:03:09.784528 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:09.784266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:44335)
0808 07:03:09.784440 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:09.784479 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260808 07:03:09.825481 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:09.825266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:47555)
0808 07:03:09.825406 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:09.825440 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260808 07:03:09.966429 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:09.966224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:40319)
0808 07:03:09.966350 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:09.966388 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260808 07:03:10.785259 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:10.785034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:48297)
0808 07:03:10.785173 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:10.785213 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260808 07:03:10.826115 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:10.825921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:59851)
0808 07:03:10.826040 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:10.826074 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260808 07:03:10.967134 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:10.966859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:60011)
0808 07:03:10.967055 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:10.967089 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":127,"threads_started":1}
I20260808 07:03:11.238709 14843 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:11.238958 14843 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:11.239177 14843 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
W20260808 07:03:11.239542 14253 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
W20260808 07:03:11.239638 14252 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:11.239704 14252 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3; no voters: 0192258a7d7d48e5a55ccc3f3a156edc, 9be22706cc4a4fa9b8c8597ef794e1c0
I20260808 07:03:11.239797 14843 raft_consensus.cc:2726] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:03:11.786103 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:11.785863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:52689)
0808 07:03:11.786019 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:11.786054 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260808 07:03:11.826750 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:11.826563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:33169)
0808 07:03:11.826678 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:11.826709 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260808 07:03:11.967794 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:11.967576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:60905)
0808 07:03:11.967719 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:11.967753 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260808 07:03:12.786840 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:12.786603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:52429)
0808 07:03:12.786749 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:12.786790 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260808 07:03:12.827412 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:12.827166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:51799)
0808 07:03:12.827336 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:12.827369 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260808 07:03:12.968468 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:12.968247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:58005)
0808 07:03:12.968392 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:12.968424 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260808 07:03:13.787741 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:13.787466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:34043)
0808 07:03:13.787628 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:13.787685 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260808 07:03:13.828120 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:13.827898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:37679)
0808 07:03:13.828032 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:13.828073 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260808 07:03:13.969161 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:13.968929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:38599)
0808 07:03:13.969077 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:13.969114 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 07:03:14.788457 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:14.788206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:52799)
0808 07:03:14.788366 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:14.788409 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260808 07:03:14.828838 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:14.828591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:50287)
0808 07:03:14.828761 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:14.828794 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260808 07:03:14.969900 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:14.969671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:46343)
0808 07:03:14.969808 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:14.969852 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260808 07:03:15.739138 14860 raft_consensus.cc:482] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:15.739238 14860 raft_consensus.cc:504] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0192258a7d7d48e5a55ccc3f3a156edc" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42781 } } peers { permanent_uuid: "9be22706cc4a4fa9b8c8597ef794e1c0" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 43901 } } peers { permanent_uuid: "4245b35c807249edb2caab8580bc2cf3" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 37829 } }
I20260808 07:03:15.739408 14860 leader_election.cc:288] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781), 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901)
W20260808 07:03:15.739794 14253 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0192258a7d7d48e5a55ccc3f3a156edc (127.24.181.67:42781): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
W20260808 07:03:15.739866 14252 leader_election.cc:341] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9be22706cc4a4fa9b8c8597ef794e1c0 (127.24.181.66:43901): Not found: Tablet not found: 6d6cc844ea854a698677b20586a77eef
I20260808 07:03:15.739934 14252 leader_election.cc:302] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4245b35c807249edb2caab8580bc2cf3; no voters: 0192258a7d7d48e5a55ccc3f3a156edc, 9be22706cc4a4fa9b8c8597ef794e1c0
I20260808 07:03:15.740026 14860 raft_consensus.cc:2726] T 6d6cc844ea854a698677b20586a77eef P 4245b35c807249edb2caab8580bc2cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:03:15.789274 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:15.789063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.65:56207)
0808 07:03:15.789192 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:15.789230 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260808 07:03:15.829530 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:15.829336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.66:42173)
0808 07:03:15.829453 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:15.829488 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 07:03:15.967476 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 14184
I20260808 07:03:15.973452 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:15.971315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34431 (local address 127.24.181.67:34565)
0808 07:03:15.973356 (+ 2041us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:15.973399 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2022,"thread_start_us":963,"threads_started":1}
I20260808 07:03:15.984189 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 14426
I20260808 07:03:15.993831 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 14607
2026-08-08T07:03:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18756 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:03:16 dist-test-slave-nfb5 krb5kdc[14868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:03:16 dist-test-slave-nfb5 krb5kdc[14868](info): set up 2 sockets
Aug 08 07:03:16 dist-test-slave-nfb5 krb5kdc[14868](info): commencing operation
krb5kdc: starting...
W20260808 07:03:17.222007 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:03:17.345459 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:17.428061 14889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:17.428318 14889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:17.428359 14889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:17.429519 14889 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:03:17.429592 14889 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:03:17.429613 14889 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:03:17.429634 14889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:03:17.429662 14889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:17.429695 14889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:17.429723 14889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:03:17.429765 14889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:03:17.431304 14889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:40205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:17.431589 14889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:17.431854 14889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:17.437209 14899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.437258 14889 server_base.cc:1034] running on GCE node
W20260808 07:03:17.437209 14895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.437201 14896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.437727 14889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:17.437994 14889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:17.439134 14889 hybrid_clock.cc:648] HybridClock initialized: now 1786172597439114 us; error 45 us; skew 500 ppm
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:17.441648 14889 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:03:17.442144 14889 webserver.cc:460] Webserver started at http://127.24.181.126:43431/ using document root <none> and password file <none>
I20260808 07:03:17.442404 14889 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:17.442453 14889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:17.442613 14889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:03:17.443791 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "3d7f0877cabf4d2fa7c62bae42bafeed"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "ac291cab52226c6477a7d4ad2af2729e"
server_key_iv: "d3a1d9bb1cf412b7db17c2ae6e8d59cd"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.444167 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "3d7f0877cabf4d2fa7c62bae42bafeed"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "ac291cab52226c6477a7d4ad2af2729e"
server_key_iv: "d3a1d9bb1cf412b7db17c2ae6e8d59cd"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.445542 14889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:03:17.446297 14911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.446514 14889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:17.446612 14889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "3d7f0877cabf4d2fa7c62bae42bafeed"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "ac291cab52226c6477a7d4ad2af2729e"
server_key_iv: "d3a1d9bb1cf412b7db17c2ae6e8d59cd"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.446688 14889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:17.455946 14889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:17.458562 14889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:17.458774 14889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:17.463038 14889 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:40205
I20260808 07:03:17.463160 14979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:40205 every 8 connection(s)
I20260808 07:03:17.463569 14889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:03:17.464433 14980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:17.466632 14980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap starting.
I20260808 07:03:17.467221 14980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:17.467482 14980 log.cc:824] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:17.468184 14980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: No bootstrap required, opened a new log
I20260808 07:03:17.469732 14980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:17.469887 14980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:17.469913 14980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Initialized, Role: FOLLOWER
I20260808 07:03:17.470053 14980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:17.470137 14980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:03:17.470199 14980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:03:17.470289 14980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:17.470438 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 14889
I20260808 07:03:17.470551 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:03:17.470753 25301 external_mini_cluster.cc:1428] Setting key 860336817808464e5d8dfe8700d858b4
I20260808 07:03:17.471097 14980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:17.471236 14980 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7f0877cabf4d2fa7c62bae42bafeed; no voters:
I20260808 07:03:17.471414 14980 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:03:17.471670 14990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:17.471829 14990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 LEADER]: Becoming Leader. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Running, Role: LEADER
I20260808 07:03:17.471841 14980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:03:17.471984 14990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:17.472767 14992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f0877cabf4d2fa7c62bae42bafeed. Latest consensus state: current_term: 1 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:17.472886 14991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:17.472893 14992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:17.472970 14991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:17.473225 15001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:03:17.473848 15001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:03:17.475449 15001 catalog_manager.cc:1347] Generated new cluster ID: 712a4ce1c3564e1289de104ddfc1060b
I20260808 07:03:17.475502 15001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:17.483640 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:17.471270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.24.181.126:40205)
0808 07:03:17.471488 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:17.471499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:17.471535 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:03:17.471679 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:17.471687 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:17.471740 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:17.472542 (+ 802us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:17.472785 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.474308 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.475378 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.475956 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.476058 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:17.479151 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:17.479155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:17.479155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:17.479168 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:17.481856 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.482293 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.482296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.482297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.482408 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.482647 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.482648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.482648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.482991 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:17.483135 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:17.483285 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:03:17.483418 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260808 07:03:17.498106 15001 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:03:17.498876 15001 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:03:17.502453 15001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Generated new TSK 0
I20260808 07:03:17.502641 15001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:03:17.537657 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:17.639460 15014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:17.639688 15014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:17.639729 15014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:17.640892 15014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:17.640959 15014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:17.641005 15014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:03:17.642462 15014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:17.642709 15014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:17.642997 15014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:17.647979 15024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.647992 15023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.647980 15026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.647996 15014 server_base.cc:1034] running on GCE node
I20260808 07:03:17.648402 15014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:17.648590 15014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:17.649757 15014 hybrid_clock.cc:648] HybridClock initialized: now 1786172597649743 us; error 30 us; skew 500 ppm
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:17.652401 15014 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:03:17.652995 15014 webserver.cc:460] Webserver started at http://127.24.181.65:46497/ using document root <none> and password file <none>
I20260808 07:03:17.653260 15014 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:17.653327 15014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:17.653475 15014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:03:17.654495 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "b0310a32cfd4f84e3954fc83b7c42001"
server_key_iv: "f37018dc7508e54896cb28541850110b"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.654872 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "b0310a32cfd4f84e3954fc83b7c42001"
server_key_iv: "f37018dc7508e54896cb28541850110b"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.656279 15014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:17.657073 15037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.657269 15014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:17.657377 15014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "b0310a32cfd4f84e3954fc83b7c42001"
server_key_iv: "f37018dc7508e54896cb28541850110b"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.657465 15014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:17.666014 15014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:17.667943 15014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:17.668105 15014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:17.668334 15014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:17.668639 15014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:17.668696 15014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.668737 15014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:17.668766 15014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.675136 15014 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:44019
I20260808 07:03:17.675190 15192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:44019 every 8 connection(s)
I20260808 07:03:17.675490 15014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:17.683611 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15014
I20260808 07:03:17.683748 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:03:17.683955 25301 external_mini_cluster.cc:1428] Setting key 9a1b2018e5fed264137ed6a99dee0a2b
I20260808 07:03:17.684587 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:17.676176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:53287 (local address 127.24.181.126:40205)
0808 07:03:17.676247 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:17.676249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:17.676426 (+ 177us) server_negotiation.cc:407] Connection header received
0808 07:03:17.676886 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:17.676887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:17.676898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:17.676957 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:17.677908 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.678394 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.679427 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.679629 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.679688 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:17.681826 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:17.681827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:17.681828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:17.681835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:17.683120 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.683487 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.683488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.683489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.683555 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.683873 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.683874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.683875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.684020 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:17.684088 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:17.684403 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 07:03:17.684497 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:03:17.684798 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:17.676127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:53287)
0808 07:03:17.676298 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:17.676338 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:17.676807 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:17.677026 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:17.677030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:17.677631 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:17.677831 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:17.677836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.678476 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.678477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:17.679351 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:17.679354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.679712 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.679713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:17.679829 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:17.680332 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:17.680340 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:17.681699 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:17.683188 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:17.683195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:17.683196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:17.683413 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:17.683625 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:17.683626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:17.683626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:17.683800 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:17.684078 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:17.684093 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:17.684265 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:03:17.684377 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:03:17.684532 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260808 07:03:17.685256 15193 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:17.685365 15193 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:17.685534 15193 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:17.686101 14936 ts_manager.cc:194] Registered new tserver with Master: 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019)
I20260808 07:03:17.686854 14936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:53287
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:03:17.709460 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:17.792518 15200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:17.792755 15200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:17.792799 15200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:17.793922 15200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:17.793998 15200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:17.794044 15200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:03:17.795579 15200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:17.795842 15200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:17.796065 15200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:17.801097 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.801086 15209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.801298 15200 server_base.cc:1034] running on GCE node
W20260808 07:03:17.801086 15208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.801586 15200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:17.801798 15200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:17.802978 15200 hybrid_clock.cc:648] HybridClock initialized: now 1786172597802963 us; error 30 us; skew 500 ppm
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:17.805469 15200 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:03:17.805857 15200 webserver.cc:460] Webserver started at http://127.24.181.66:34827/ using document root <none> and password file <none>
I20260808 07:03:17.806061 15200 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:17.806102 15200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:17.806191 15200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:03:17.807245 15200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "7c732d109216457fb73cc044399c939b"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "a5985d9c675870b32790dc2cf54f8334"
server_key_iv: "98a925a5ef41cac2fcdb36f0c9919895"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.807552 15200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "7c732d109216457fb73cc044399c939b"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "a5985d9c675870b32790dc2cf54f8334"
server_key_iv: "98a925a5ef41cac2fcdb36f0c9919895"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.808881 15200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:03:17.809577 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.809769 15200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:03:17.809876 15200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "7c732d109216457fb73cc044399c939b"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "a5985d9c675870b32790dc2cf54f8334"
server_key_iv: "98a925a5ef41cac2fcdb36f0c9919895"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.809962 15200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:17.827386 15200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:17.829257 15200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:17.829432 15200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:17.829679 15200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:17.830010 15200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:17.830068 15200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.830112 15200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:17.830158 15200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.836472 15200 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:42563
I20260808 07:03:17.836524 15383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:42563 every 8 connection(s)
I20260808 07:03:17.836853 15200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:17.844406 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15200
I20260808 07:03:17.844539 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:03:17.844769 25301 external_mini_cluster.cc:1428] Setting key 8fb277b64d725a990dbaf606df65a91e
I20260808 07:03:17.846696 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:17.837562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:54079)
0808 07:03:17.837762 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:17.837800 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:17.838316 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:17.838589 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:17.838592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:17.839236 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:17.839448 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:17.839453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.840176 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.840178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:17.841000 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:17.841002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.841412 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.841413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:17.841539 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:17.841998 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:17.842006 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:17.843423 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:17.845243 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:17.845252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:17.845254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:17.845601 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:17.845819 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:17.845819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:17.845819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:17.845920 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:17.846216 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:17.846221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:17.846333 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 07:03:17.846417 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:03:17.846536 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260808 07:03:17.846704 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:17.837613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:54079 (local address 127.24.181.126:40205)
0808 07:03:17.837707 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:17.837710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:17.837890 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:03:17.838440 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:17.838441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:17.838456 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:17.838504 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:17.839555 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.840070 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.841098 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:17.841315 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:17.841402 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:17.843526 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:17.843528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:17.843528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:17.843537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:17.845139 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.845677 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.845678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.845679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.845749 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:17.845984 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:17.845985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:17.845985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:17.846134 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:17.846204 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:17.846465 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:03:17.846563 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:03:17.847689 15384 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:17.847803 15384 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:17.848006 15384 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:17.848380 14936 ts_manager.cc:194] Registered new tserver with Master: 7c732d109216457fb73cc044399c939b (127.24.181.66:42563)
I20260808 07:03:17.848807 14936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:54079
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:03:17.873504 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:17.958349 15390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:17.958573 15390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:17.958612 15390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:17.959810 15390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:17.959865 15390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:17.959908 15390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:03:17.961469 15390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:17.961710 15390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:17.961946 15390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:17.966874 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.966874 15401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:17.966975 15398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:17.967121 15390 server_base.cc:1034] running on GCE node
I20260808 07:03:17.967425 15390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:17.967620 15390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:17.968792 15390 hybrid_clock.cc:648] HybridClock initialized: now 1786172597968780 us; error 29 us; skew 500 ppm
Aug 08 07:03:17 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:17.971315 15390 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:03:17.971804 15390 webserver.cc:460] Webserver started at http://127.24.181.67:34355/ using document root <none> and password file <none>
I20260808 07:03:17.972048 15390 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:17.972117 15390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:17.972273 15390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:03:17.973378 15390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "be34d152b57243d4b74bf62bf36b9a18"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "7972fec3ad0a7bbc11d831416490e938"
server_key_iv: "d9e373337f7474fb347673589dfcd351"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.973788 15390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "be34d152b57243d4b74bf62bf36b9a18"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "7972fec3ad0a7bbc11d831416490e938"
server_key_iv: "d9e373337f7474fb347673589dfcd351"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.975152 15390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:03:17.975827 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.976004 15390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.976111 15390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "be34d152b57243d4b74bf62bf36b9a18"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "7972fec3ad0a7bbc11d831416490e938"
server_key_iv: "d9e373337f7474fb347673589dfcd351"
server_key_version: "encryptionkey@0"
I20260808 07:03:17.976186 15390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:17.995241 15390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:17.997612 15390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:17.997783 15390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:17.998026 15390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:17.998333 15390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:17.998389 15390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:17.998436 15390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:17.998461 15390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:18.004871 15390 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41325
I20260808 07:03:18.004927 15569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41325 every 8 connection(s)
I20260808 07:03:18.005298 15390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:18.008244 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15390
I20260808 07:03:18.008342 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:03:18.008502 25301 external_mini_cluster.cc:1428] Setting key 5358d4e9872051963bf21b6b4ebac312
Aug 08 07:03:18 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:18.015041 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.006101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:40117 (local address 127.24.181.126:40205)
0808 07:03:18.006177 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.006179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.006373 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:03:18.006879 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.006879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.006889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.006996 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:18.007929 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.008687 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.009993 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.010182 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.010239 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.012478 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:18.012479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:18.012480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:18.012486 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:18.013821 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:18.014180 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:18.014181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:18.014181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:18.014232 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:18.014449 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:18.014449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:18.014449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:18.014547 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:18.014599 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.014856 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.014976 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:03:18.015219 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.006066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:40117)
0808 07:03:18.006242 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.006284 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.006793 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.007070 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.007074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.007643 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:18.007847 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.007851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.008829 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.008830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.009910 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.009913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.010260 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.010261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.010361 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.010969 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:18.010980 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:18.012390 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:18.013892 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:18.013899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:18.013901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:18.014114 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:18.014293 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:18.014293 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:18.014293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:18.014393 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:18.014602 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:18.014607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:18.014719 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.014813 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.014986 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260808 07:03:18.015856 15570 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:18.015971 15570 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:18.016152 15570 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:18.016568 14936 ts_manager.cc:194] Registered new tserver with Master: be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
I20260808 07:03:18.016942 14936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:40117
I20260808 07:03:18.020335 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:03:18.027197 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.021620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.24.181.126:40205)
0808 07:03:18.021678 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.021680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.021781 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:03:18.021821 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.021822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.021832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.021884 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:18.022417 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.022853 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.023624 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.023798 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.023854 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.025037 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:18.025038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:18.025038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:18.025043 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:18.026169 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:18.026480 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:18.026481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:18.026481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:18.026531 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:18.026702 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:18.026703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:18.026703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:18.026832 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:18.026906 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.027078 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.027137 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:03:18.028697 14936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:03:18.030280 14936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:03:18.036872 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.032792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42563 (local address 127.0.0.1:36770)
0808 07:03:18.033314 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.033325 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.033383 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.033710 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.033711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.033713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:18.033760 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.033761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.035052 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.035053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.035846 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.035849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.036644 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.036645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.036720 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.036731 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.036786 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.036820 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
I20260808 07:03:18.036952 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.033285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.24.181.66:42563)
0808 07:03:18.033509 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.033513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.033523 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:03:18.033579 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.033581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.033596 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.033706 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:18.033821 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.034890 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.035944 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.036565 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.036645 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.036698 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.036820 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.036864 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260808 07:03:18.037369 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.032787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.0.0.1:41546)
0808 07:03:18.032895 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.032932 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.032998 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.033378 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.033382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.033384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:18.033479 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.033481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.035002 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.035003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.036212 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.036213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.037148 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.037149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.037211 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.037214 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.037268 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.037307 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":7,"thread_start_us":59,"threads_started":1}
I20260808 07:03:18.037425 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.032922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41546 (local address 127.24.181.65:44019)
0808 07:03:18.033187 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.033191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.033201 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:03:18.033243 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.033245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.033262 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.033595 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:18.033690 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.034865 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.036305 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.037064 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.037147 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.037203 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.037290 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.037336 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260808 07:03:18.037608 15284 tablet_service.cc:1467] Processing CreateTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 (DEFAULT_TABLE table=test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:18.037983 15284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aeee8e10f4457bbec9b660a1084d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:18.038348 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.033032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41325 (local address 127.0.0.1:36534)
0808 07:03:18.033327 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.033337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.033383 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.033837 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.033838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.033840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:18.033877 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.033879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.034834 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.034835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.036210 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.036213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.038109 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.038110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.038173 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.038178 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.038238 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.038278 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":35,"thread_start_us":73,"threads_started":1}
I20260808 07:03:18.038421 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.033136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.24.181.67:41325)
0808 07:03:18.033502 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.033506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.033514 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:03:18.033562 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.033564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.033581 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.033763 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:18.033963 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.034713 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.037239 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.037937 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.038012 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.038068 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.038268 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.038317 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":84,"threads_started":1}
I20260808 07:03:18.038375 15099 tablet_service.cc:1467] Processing CreateTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 (DEFAULT_TABLE table=test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:18.038757 15099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aeee8e10f4457bbec9b660a1084d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:18.039650 15483 tablet_service.cc:1467] Processing CreateTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 (DEFAULT_TABLE table=test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:18.039997 15483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71aeee8e10f4457bbec9b660a1084d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:18.041769 15593 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap starting.
I20260808 07:03:18.041797 15594 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Bootstrap starting.
I20260808 07:03:18.042374 15593 tablet_bootstrap.cc:654] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:18.042466 15594 tablet_bootstrap.cc:654] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:18.042958 15593 log.cc:824] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:18.043071 15594 log.cc:824] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:18.043439 15599 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Bootstrap starting.
I20260808 07:03:18.043740 15593 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: No bootstrap required, opened a new log
I20260808 07:03:18.043797 15593 ts_tablet_manager.cc:1397] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:18.043848 15594 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: No bootstrap required, opened a new log
I20260808 07:03:18.043913 15594 ts_tablet_manager.cc:1397] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:18.045318 15593 raft_consensus.cc:348] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.045462 15593 raft_consensus.cc:374] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:18.045485 15593 raft_consensus.cc:729] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e2c167b6d4647b52adbd075a33cd8, State: Initialized, Role: FOLLOWER
I20260808 07:03:18.045574 15593 consensus_queue.cc:260] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.045575 15594 raft_consensus.cc:348] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.045768 15594 raft_consensus.cc:374] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:18.045814 15594 raft_consensus.cc:729] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c732d109216457fb73cc044399c939b, State: Initialized, Role: FOLLOWER
I20260808 07:03:18.045840 15593 ts_tablet_manager.cc:1428] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:18.045851 15193 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
I20260808 07:03:18.045928 15594 consensus_queue.cc:260] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.046192 15594 ts_tablet_manager.cc:1428] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:03:18.046252 15384 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
I20260808 07:03:18.047379 15599 tablet_bootstrap.cc:654] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:18.047801 15599 log.cc:824] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:18.048460 15599 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: No bootstrap required, opened a new log
I20260808 07:03:18.048516 15599 ts_tablet_manager.cc:1397] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:03:18.050010 15599 raft_consensus.cc:348] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.050139 15599 raft_consensus.cc:374] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:18.050194 15599 raft_consensus.cc:729] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be34d152b57243d4b74bf62bf36b9a18, State: Initialized, Role: FOLLOWER
I20260808 07:03:18.050309 15599 consensus_queue.cc:260] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.050556 15599 ts_tablet_manager.cc:1428] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:03:18.050640 15570 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
W20260808 07:03:18.087508 15385 tablet.cc:2369] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:03:18.176193 15194 tablet.cc:2369] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:03:18.255914 15571 tablet.cc:2369] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:03:18.296977 15601 raft_consensus.cc:482] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:18.297127 15601 raft_consensus.cc:504] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.297534 15601 leader_election.cc:288] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
I20260808 07:03:18.300896 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.297927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60397 (local address 127.24.181.67:41325)
0808 07:03:18.297978 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.297981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.297993 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:03:18.298026 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.298027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.298037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.298110 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:18.298337 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.298876 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.299992 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.300382 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.300439 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.300446 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.300771 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.300812 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:03:18.300902 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.297789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41325 (local address 127.24.181.66:60397)
0808 07:03:18.297885 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.297900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.297959 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.298163 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.298164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.298167 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:18.298254 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.298256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.298999 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.299000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.299889 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.299892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.300561 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.300570 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.300684 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.300688 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.300751 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.300794 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":1}
I20260808 07:03:18.301005 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.297930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:49509 (local address 127.24.181.65:44019)
0808 07:03:18.297991 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.297993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.298157 (+ 164us) server_negotiation.cc:407] Connection header received
0808 07:03:18.298197 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.298198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.298210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.298280 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:18.298481 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.299095 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.300056 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.300590 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.300686 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.300698 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.300872 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.300924 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:03:18.301186 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.297842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.24.181.66:49509)
0808 07:03:18.298058 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.298068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:18.298131 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:18.298312 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:18.298312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:18.298314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:18.298401 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.298402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.299179 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.299179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.299921 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:18.299923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.300693 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.300693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:18.300778 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.300783 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:03:18.300842 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:03:18.300880 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":123,"threads_started":1}
I20260808 07:03:18.301397 15508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aeee8e10f4457bbec9b660a1084d67" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be34d152b57243d4b74bf62bf36b9a18" is_pre_election: true
I20260808 07:03:18.301584 15508 raft_consensus.cc:2454] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 0.
I20260808 07:03:18.301571 15127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aeee8e10f4457bbec9b660a1084d67" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" is_pre_election: true
I20260808 07:03:18.301738 15127 raft_consensus.cc:2454] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 0.
I20260808 07:03:18.301839 15223 leader_election.cc:302] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18; no voters:
I20260808 07:03:18.302055 15601 raft_consensus.cc:2781] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:03:18.302120 15601 raft_consensus.cc:482] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:03:18.302167 15601 raft_consensus.cc:3037] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:18.303033 15601 raft_consensus.cc:504] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.303210 15601 leader_election.cc:288] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 election: Requested vote from peers 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
I20260808 07:03:18.303388 15127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aeee8e10f4457bbec9b660a1084d67" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
I20260808 07:03:18.303417 15508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71aeee8e10f4457bbec9b660a1084d67" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be34d152b57243d4b74bf62bf36b9a18"
I20260808 07:03:18.303479 15127 raft_consensus.cc:3037] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:18.303516 15508 raft_consensus.cc:3037] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:18.304279 15127 raft_consensus.cc:2454] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 1.
I20260808 07:03:18.304281 15508 raft_consensus.cc:2454] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 1.
I20260808 07:03:18.304508 15224 leader_election.cc:302] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8, 7c732d109216457fb73cc044399c939b; no voters:
I20260808 07:03:18.304622 15601 raft_consensus.cc:2781] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:18.304814 15601 raft_consensus.cc:686] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 1 LEADER]: Becoming Leader. State: Replica: 7c732d109216457fb73cc044399c939b, State: Running, Role: LEADER
I20260808 07:03:18.304922 15601 consensus_queue.cc:237] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:18.305840 14936 catalog_manager.cc:5495] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c732d109216457fb73cc044399c939b (127.24.181.66). New cstate: current_term: 1 leader_uuid: "7c732d109216457fb73cc044399c939b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } health_report { overall_health: UNKNOWN } } }
I20260808 07:03:18.319727 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.317130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.24.181.66:42563)
0808 07:03:18.317197 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:18.317200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:18.317208 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:03:18.317247 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:18.317248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:18.317265 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:18.317342 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:18.317497 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.317966 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.318830 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:18.319070 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:18.319154 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:18.319360 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:18.319477 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:18.319567 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:18.319607 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:03:18.319628 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:03:18.321566 15127 raft_consensus.cc:1261] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Refusing update from remote peer 7c732d109216457fb73cc044399c939b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:18.321566 15508 raft_consensus.cc:1261] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Refusing update from remote peer 7c732d109216457fb73cc044399c939b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:18.321969 15601 consensus_queue.cc:1035] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:18.322134 15611 consensus_queue.cc:1035] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [LEADER]: Connected to new peer: Peer: permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:18.324659 15615 mvcc.cc:205] Tried to move back new op lower bound from 7316162962722631680 to 7316162962657472512. Current Snapshot: MvccSnapshot[applied={T|T < 7316162962722631680}]
I20260808 07:03:18.325054 15617 mvcc.cc:205] Tried to move back new op lower bound from 7316162962722631680 to 7316162962657472512. Current Snapshot: MvccSnapshot[applied={T|T < 7316162962722631680}]
I20260808 07:03:18.327214 14934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:18.327315 14934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:18.328024 14934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Sending DeleteTablet for 3 replicas of tablet 71aeee8e10f4457bbec9b660a1084d67
I20260808 07:03:18.328351 15284 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:18 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:36770
I20260808 07:03:18.328394 15483 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:18 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:36534
I20260808 07:03:18.328418 15099 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:18 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:41546
I20260808 07:03:18.328660 15628 tablet_replica.cc:330] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: stopping tablet replica
I20260808 07:03:18.328780 15628 raft_consensus.cc:2229] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:18.328814 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15014
I20260808 07:03:18.328899 15628 raft_consensus.cc:2258] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:18.337692 15626 tablet_replica.cc:330] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: stopping tablet replica
I20260808 07:03:18.337903 15626 raft_consensus.cc:2229] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:03:18.338054 15626 raft_consensus.cc:2258] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:18.339337 15626 ts_tablet_manager.cc:1905] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:18.341526 15626 ts_tablet_manager.cc:1918] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:18.341615 15626 log.cc:1196] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/71aeee8e10f4457bbec9b660a1084d67
I20260808 07:03:18.341919 15626 ts_tablet_manager.cc:1939] T 71aeee8e10f4457bbec9b660a1084d67 P 7c732d109216457fb73cc044399c939b: Deleting consensus metadata
I20260808 07:03:18.342403 14912 catalog_manager.cc:4841] TS 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): tablet 71aeee8e10f4457bbec9b660a1084d67 (table test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]) successfully deleted
W20260808 07:03:18.344166 14913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:44019 (error 108)
I20260808 07:03:18.344415 15627 tablet_replica.cc:330] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: stopping tablet replica
I20260808 07:03:18.344509 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15200
I20260808 07:03:18.344620 15627 raft_consensus.cc:2229] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:18.344764 15627 raft_consensus.cc:2258] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:18.345221 15627 ts_tablet_manager.cc:1905] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:18.346974 15627 ts_tablet_manager.cc:1918] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:18.347077 15627 log.cc:1196] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/71aeee8e10f4457bbec9b660a1084d67
I20260808 07:03:18.347396 15627 ts_tablet_manager.cc:1939] T 71aeee8e10f4457bbec9b660a1084d67 P be34d152b57243d4b74bf62bf36b9a18: Deleting consensus metadata
I20260808 07:03:18.347846 14912 catalog_manager.cc:4841] TS be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): tablet 71aeee8e10f4457bbec9b660a1084d67 (table test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]) successfully deleted
I20260808 07:03:18.352799 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:18.352632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.0.0.1:41556)
0808 07:03:18.352696 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:18.352755 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:03:18.352953 14913 catalog_manager.cc:4584] TS 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71aeee8e10f4457bbec9b660a1084d67: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44019: connect: Connection refused (error 111)
I20260808 07:03:18.356841 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15390
I20260808 07:03:18.367877 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 14889
I20260808 07:03:38.377679 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--webserver_interface=127.24.181.126
--webserver_port=43431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:38.464630 15633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:38.464867 15633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:38.464905 15633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:38.466035 15633 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:03:38.466100 15633 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:03:38.466122 15633 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:03:38.466142 15633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:03:38.466164 15633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:38.466194 15633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:38.466221 15633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:03:38.466248 15633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:03:38.467864 15633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:40205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=43431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:38.468231 15633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:38.468501 15633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:38.478252 15645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.478271 15648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.478332 15644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:38.478305 15633 server_base.cc:1034] running on GCE node
I20260808 07:03:38.479012 15633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:38.479409 15633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:38.480584 15633 hybrid_clock.cc:648] HybridClock initialized: now 1786172618480575 us; error 31 us; skew 500 ppm
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:38.488950 15633 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:03:38.489490 15633 webserver.cc:460] Webserver started at http://127.24.181.126:43431/ using document root <none> and password file <none>
I20260808 07:03:38.489763 15633 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:38.489861 15633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:38.491351 15633 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:03:38.492089 15658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.492282 15633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:38.492398 15633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "3d7f0877cabf4d2fa7c62bae42bafeed"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "ac291cab52226c6477a7d4ad2af2729e"
server_key_iv: "d3a1d9bb1cf412b7db17c2ae6e8d59cd"
server_key_version: "encryptionkey@0"
I20260808 07:03:38.492834 15633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:38.506644 15633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:38.508463 15633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:38.508632 15633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:38.512897 15633 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:40205
I20260808 07:03:38.512931 15729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:40205 every 8 connection(s)
I20260808 07:03:38.513319 15633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:03:38.513700 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15633
I20260808 07:03:38.514035 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:44019
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=46497
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:38.515075 15732 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:03:38.515758 15732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap starting.
I20260808 07:03:38.525844 15732 log.cc:824] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:38.527871 15732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:03:38.528228 15732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap complete.
I20260808 07:03:38.530599 15732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:38.530865 15732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Initialized, Role: FOLLOWER
I20260808 07:03:38.531073 15732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:38.531183 15732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:03:38.531243 15732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:03:38.531291 15732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:03:38.532291 15732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:38.532414 15732 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7f0877cabf4d2fa7c62bae42bafeed; no voters:
I20260808 07:03:38.532598 15732 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:03:38.532665 15737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:03:38.532822 15737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 LEADER]: Becoming Leader. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Running, Role: LEADER
I20260808 07:03:38.532888 15732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:03:38.532974 15737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:38.533236 15737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f0877cabf4d2fa7c62bae42bafeed. Latest consensus state: current_term: 2 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:38.533308 15737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:38.533238 15738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:38.533517 15738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:38.533689 15746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:03:38.534560 15746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:03:38.534812 15746 catalog_manager.cc:1259] Loaded cluster ID: 712a4ce1c3564e1289de104ddfc1060b
I20260808 07:03:38.534960 15746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:03:38.536094 15746 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:03:38.536267 15746 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Loaded TSK: 0
I20260808 07:03:38.547772 15746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Generated new TSK 1
I20260808 07:03:38.547863 15746 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:03:38.601459 15735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:38.601657 15735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:38.601681 15735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:38.602824 15735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:38.602866 15735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:38.602893 15735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:03:38.604404 15735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:44019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=46497
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:38.604624 15735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:38.604878 15735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:38.610050 15771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.610051 15776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.610121 15774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:38.610631 15735 server_base.cc:1034] running on GCE node
I20260808 07:03:38.610841 15735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:38.611125 15735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:38.612313 15735 hybrid_clock.cc:648] HybridClock initialized: now 1786172618612295 us; error 35 us; skew 500 ppm
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:38.615026 15735 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:03:38.615523 15735 webserver.cc:460] Webserver started at http://127.24.181.65:46497/ using document root <none> and password file <none>
I20260808 07:03:38.615780 15735 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:38.615844 15735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:38.617215 15735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:38.617928 15785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.618104 15735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:38.618209 15735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "b0310a32cfd4f84e3954fc83b7c42001"
server_key_iv: "f37018dc7508e54896cb28541850110b"
server_key_version: "encryptionkey@0"
I20260808 07:03:38.618552 15735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:38.634325 15735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:38.636654 15735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:38.636818 15735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:38.637092 15735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:38.637576 15794 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:03:38.639544 15735 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:03:38.639611 15735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:03:38.639652 15735 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:03:38.640312 15735 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:03:38.640370 15735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:38.640436 15794 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap starting.
I20260808 07:03:38.648007 15735 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:44019
I20260808 07:03:38.648059 15942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:44019 every 8 connection(s)
I20260808 07:03:38.648451 15735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:03:38.649945 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15735
I20260808 07:03:38.650265 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:42563
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=34827
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:38.653012 15794 log.cc:824] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:38.657760 15794 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:38.658102 15794 tablet_bootstrap.cc:492] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap complete.
I20260808 07:03:38.660064 15794 ts_tablet_manager.cc:1397] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.011s
I20260808 07:03:38.661365 15794 raft_consensus.cc:348] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:38.661680 15794 raft_consensus.cc:729] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e2c167b6d4647b52adbd075a33cd8, State: Initialized, Role: FOLLOWER
I20260808 07:03:38.661839 15794 consensus_queue.cc:260] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } } peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } }
I20260808 07:03:38.662096 15794 ts_tablet_manager.cc:1428] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:03:38.663959 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.649322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:52453 (local address 127.24.181.126:40205)
0808 07:03:38.650531 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.650541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.650581 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:03:38.650712 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.650718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.650777 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.650905 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:38.651919 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.653012 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.654661 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.655346 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.655430 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.658784 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:38.658789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:38.658790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:38.658803 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:38.661206 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.662092 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.662095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.662095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.662278 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.662611 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.662612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.662613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.662995 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:38.663066 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.663520 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.663686 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1153,"thread_start_us":83,"threads_started":1}
I20260808 07:03:38.665125 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.649245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:52453)
0808 07:03:38.649518 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.649563 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.650277 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.651074 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.651078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.651486 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:38.651766 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.651773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.653193 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.653195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.654555 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.654558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.655586 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.655588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.655738 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.656393 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:38.656405 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:38.658615 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:38.661377 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.661384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.661387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.661977 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.662377 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.662378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.662379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.662522 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.663126 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:38.663134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:38.663302 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.664714 (+ 1412us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.664930 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":106,"threads_started":1}
I20260808 07:03:38.666105 15943 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:38.666255 15943 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:38.666535 15943 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:38.667367 15677 ts_manager.cc:194] Registered new tserver with Master: 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019)
I20260808 07:03:38.668570 15677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:52453
I20260808 07:03:38.669479 15943 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.65@KRBTEST.COM
I20260808 07:03:38.677675 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.668261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.24.181.65:44019)
0808 07:03:38.668605 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.668609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.668617 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:03:38.668652 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.668653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.668667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.668723 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:38.669427 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.670621 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.671742 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.672114 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.672200 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.674633 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:38.674642 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:38.674642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:38.674649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:38.676313 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.676765 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.676769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.676769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.676833 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.677042 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.677043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.677044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.677203 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:38.677298 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.677490 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.677583 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":63,"threads_started":1}
I20260808 07:03:38.677687 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.668012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.0.0.1:51844)
0808 07:03:38.668399 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.668432 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.668494 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.668780 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.668785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.669279 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:38.669336 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.669338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.670702 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.670703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.671645 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.671648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.672214 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.672215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.672297 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.672799 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:38.672804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:38.674538 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:38.676415 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.676422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.676424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.676656 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.676911 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.676912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.676912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.676978 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.677289 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:38.677296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:38.677349 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.677450 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.677576 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":87,"threads_started":1}
I20260808 07:03:38.678154 15852 tablet_service.cc:1514] Processing DeleteTablet for tablet 71aeee8e10f4457bbec9b660a1084d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:18 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:51844
I20260808 07:03:38.678733 15955 tablet_replica.cc:330] stopping tablet replica
I20260808 07:03:38.678848 15955 raft_consensus.cc:2229] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:38.678980 15955 raft_consensus.cc:2258] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:38.679359 15955 ts_tablet_manager.cc:1905] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:38.681438 15955 ts_tablet_manager.cc:1918] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:38.681532 15955 log.cc:1196] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal/wals/71aeee8e10f4457bbec9b660a1084d67
I20260808 07:03:38.681767 15955 ts_tablet_manager.cc:1939] T 71aeee8e10f4457bbec9b660a1084d67 P 2d4e2c167b6d4647b52adbd075a33cd8: Deleting consensus metadata
I20260808 07:03:38.682250 15660 catalog_manager.cc:4841] TS 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019): tablet 71aeee8e10f4457bbec9b660a1084d67 (table test-table [id=438a07dd0bcd4d5e9a613651e4f9abee]) successfully deleted
W20260808 07:03:38.743119 15949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:38.743326 15949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:38.743352 15949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:38.744484 15949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:38.744525 15949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:38.744577 15949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:03:38.746018 15949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:42563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=34827
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:38.746244 15949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:38.746495 15949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:38.751735 15963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.751784 15962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.751816 15965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:38.751786 15949 server_base.cc:1034] running on GCE node
I20260808 07:03:38.752149 15949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:38.752334 15949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:38.753508 15949 hybrid_clock.cc:648] HybridClock initialized: now 1786172618753494 us; error 45 us; skew 500 ppm
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:38.756214 15949 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:03:38.756649 15949 webserver.cc:460] Webserver started at http://127.24.181.66:34827/ using document root <none> and password file <none>
I20260808 07:03:38.756896 15949 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:38.756982 15949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:38.758452 15949 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:03:38.759116 15973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.759305 15949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:38.759384 15949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "7c732d109216457fb73cc044399c939b"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "a5985d9c675870b32790dc2cf54f8334"
server_key_iv: "98a925a5ef41cac2fcdb36f0c9919895"
server_key_version: "encryptionkey@0"
I20260808 07:03:38.759768 15949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:38.777424 15949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:38.779421 15949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:38.779580 15949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:38.779798 15949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:38.780097 15949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:38.780130 15949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.780151 15949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:38.780198 15949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.785972 15949 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:42563
I20260808 07:03:38.786072 16129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:42563 every 8 connection(s)
I20260808 07:03:38.786384 15949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:03:38.788197 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 15949
I20260808 07:03:38.788570 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:41325
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=34355
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:38.798332 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.787165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:37333 (local address 127.24.181.126:40205)
0808 07:03:38.787251 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.787254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.787409 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:03:38.787880 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.787881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.787891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.787930 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:38.789321 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.789810 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.791079 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.791485 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.791554 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.794633 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:38.794635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:38.794635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:38.794643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:38.796663 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.797205 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.797206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.797207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.797266 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.797573 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.797574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.797574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.797709 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:38.797756 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.798063 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.798212 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:03:38.798573 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.787125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:37333)
0808 07:03:38.787287 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.787323 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.787778 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.788058 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.788063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.788951 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:38.789225 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.789232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.789907 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.789908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.790891 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.790893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.791581 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.791583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.791768 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.792394 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:38.792424 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:38.794529 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:38.796780 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.796801 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.796803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.797110 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.797350 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.797351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.797351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.797485 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.797796 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:38.797802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:38.797913 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.798024 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.798194 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":102,"threads_started":1}
I20260808 07:03:38.799177 16130 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:38.799281 16130 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:38.799499 16130 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:38.799942 15677 ts_manager.cc:194] Registered new tserver with Master: 7c732d109216457fb73cc044399c939b (127.24.181.66:42563)
I20260808 07:03:38.800629 15677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:37333
W20260808 07:03:38.874806 16133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:38.875051 16133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:38.875092 16133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:38.876303 16133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:38.876360 16133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:38.876405 16133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:03:38.877835 16133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:41325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=34355
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:38.878095 16133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:38.878322 16133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:03:38.883898 16133 server_base.cc:1034] running on GCE node
W20260808 07:03:38.883865 16140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.883882 16142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:38.884068 16139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:38.884305 16133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:38.884543 16133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:38.885710 16133 hybrid_clock.cc:648] HybridClock initialized: now 1786172618885696 us; error 45 us; skew 500 ppm
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:38.888286 16133 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:03:38.888716 16133 webserver.cc:460] Webserver started at http://127.24.181.67:34355/ using document root <none> and password file <none>
I20260808 07:03:38.888950 16133 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:38.889038 16133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:38.890453 16133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:38.891172 16156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.891355 16133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:38.891464 16133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "be34d152b57243d4b74bf62bf36b9a18"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "7972fec3ad0a7bbc11d831416490e938"
server_key_iv: "d9e373337f7474fb347673589dfcd351"
server_key_version: "encryptionkey@0"
I20260808 07:03:38.891799 16133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:38.906597 16133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:38.909458 16133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:38.909616 16133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:38.909883 16133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:38.910238 16133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:38.910301 16133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.910344 16133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:38.910372 16133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.918323 16133 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41325
I20260808 07:03:38.918398 16305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41325 every 8 connection(s)
I20260808 07:03:38.918797 16133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:38.924788 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 16133
Aug 08 07:03:38 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:03:38.929313 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.919663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:58449 (local address 127.24.181.126:40205)
0808 07:03:38.919711 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.919714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.919968 (+ 254us) server_negotiation.cc:407] Connection header received
0808 07:03:38.921227 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.921228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.921243 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.921298 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:38.921958 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.922572 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.923710 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.924027 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.924103 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.926336 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:38.926337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:38.926338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:38.926345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:38.927827 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.928274 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.928274 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.928275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.928350 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.928651 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.928652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.928653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.928792 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:38.928862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.929182 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.929249 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:03:38.929610 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.919669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:58449)
0808 07:03:38.919843 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.919879 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.921092 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.921414 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.921417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.921671 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:38.921853 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.921857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.922667 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.922669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.923606 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.923609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.924116 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.924117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.924250 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.924820 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:38.924829 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:38.926262 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:38.927914 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.927922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.927924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.928197 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.928427 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:38.928428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:38.928428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:38.928579 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:38.928862 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:38.928871 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:38.929035 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.929162 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.929350 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260808 07:03:38.930142 16307 heartbeater.cc:347] Connected to a master server at 127.24.181.126:40205
I20260808 07:03:38.930274 16307 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:38.930457 16307 heartbeater.cc:511] Master 127.24.181.126:40205 requested a full tablet report, sending...
I20260808 07:03:38.930962 15677 ts_manager.cc:194] Registered new tserver with Master: be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
I20260808 07:03:38.931321 15677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:58449
I20260808 07:03:38.932596 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.925235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.24.181.126:40205)
0808 07:03:38.925511 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.925513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.925518 (+ 5us) server_negotiation.cc:407] Connection header received
0808 07:03:38.925534 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.925535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.925543 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.925610 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:38.926280 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.927068 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.928030 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.928217 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.928301 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.929765 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:38.929766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:38.929767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:38.929774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:38.931446 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.931840 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.931841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.931841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.931904 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:38.932096 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:38.932097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:38.932097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:38.932232 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:38.932438 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.932470 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.932535 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":58,"threads_started":1}
I20260808 07:03:38.933434 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:03:38.936023 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.933725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.24.181.126:40205)
0808 07:03:38.933760 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.933761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.933907 (+ 146us) server_negotiation.cc:407] Connection header received
0808 07:03:38.933939 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.933940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.933966 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.934015 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:38.934102 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.934600 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.935391 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.935570 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.935621 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.935777 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:38.935872 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:38.935930 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.935962 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.935980 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:03:38.936254 15677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:03:38.937836 15677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:03:38.940237 15852 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8335a294df4db181969b136a425ad9 (DEFAULT_TABLE table=test-table [id=ef287680986443b09b8bee5020fd9bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:38.940521 15852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8335a294df4db181969b136a425ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:38.942505 15794 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap starting.
I20260808 07:03:38.942852 15794 tablet_bootstrap.cc:654] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:38.943881 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.939714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42563 (local address 127.0.0.1:41738)
0808 07:03:38.939769 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.939781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.939838 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.940310 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.940311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.940313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:38.940354 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.940355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.941606 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.941607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.942722 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.942724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.943581 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.943582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.943660 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.943684 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.943756 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.943806 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:03:38.943913 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.939918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.24.181.66:42563)
0808 07:03:38.940138 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.940144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.940159 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:03:38.940193 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.940195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.940209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.940276 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:38.940450 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.941500 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.942803 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.943509 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.943584 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.943637 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.943784 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.943832 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":115,"threads_started":1}
I20260808 07:03:38.944259 15794 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: No bootstrap required, opened a new log
I20260808 07:03:38.944274 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.939819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41325 (local address 127.0.0.1:40462)
0808 07:03:38.939963 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.939973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.940015 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.940906 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.940908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.940910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:38.940957 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.940959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.942095 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.942096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.943127 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.943131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.944071 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.944072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.944136 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.944140 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.944171 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.944211 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260808 07:03:38.944335 15794 ts_tablet_manager.cc:1397] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:03:38.944494 15794 raft_consensus.cc:348] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.944586 15794 raft_consensus.cc:374] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:38.944610 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.939969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.24.181.67:41325)
0808 07:03:38.940502 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.940507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.940518 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:03:38.940553 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.940555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.940570 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.940806 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:38.941043 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.942012 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.943233 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.943960 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.944396 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.944461 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.944486 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.944528 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":87,"threads_started":1}
I20260808 07:03:38.944639 15794 raft_consensus.cc:729] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e2c167b6d4647b52adbd075a33cd8, State: Initialized, Role: FOLLOWER
I20260808 07:03:38.944692 15794 consensus_queue.cc:260] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.944594 16035 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8335a294df4db181969b136a425ad9 (DEFAULT_TABLE table=test-table [id=ef287680986443b09b8bee5020fd9bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:38.944805 15794 ts_tablet_manager.cc:1428] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:38.944985 16035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8335a294df4db181969b136a425ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:38.945552 16215 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8335a294df4db181969b136a425ad9 (DEFAULT_TABLE table=test-table [id=ef287680986443b09b8bee5020fd9bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:38.945878 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8335a294df4db181969b136a425ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:38.948732 16326 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Bootstrap starting.
I20260808 07:03:38.949457 16327 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Bootstrap starting.
I20260808 07:03:38.949550 16326 tablet_bootstrap.cc:654] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:38.950067 16327 tablet_bootstrap.cc:654] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:38.950080 16326 log.cc:824] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:38.950728 16327 log.cc:824] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:38.951105 16326 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: No bootstrap required, opened a new log
I20260808 07:03:38.951203 16326 ts_tablet_manager.cc:1397] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:03:38.951572 16327 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: No bootstrap required, opened a new log
I20260808 07:03:38.951655 16327 ts_tablet_manager.cc:1397] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:03:38.953266 16326 raft_consensus.cc:348] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.953460 16326 raft_consensus.cc:374] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:38.953497 16326 raft_consensus.cc:729] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c732d109216457fb73cc044399c939b, State: Initialized, Role: FOLLOWER
I20260808 07:03:38.953625 16326 consensus_queue.cc:260] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.953649 16327 raft_consensus.cc:348] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.953835 16327 raft_consensus.cc:374] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:38.953873 16327 raft_consensus.cc:729] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be34d152b57243d4b74bf62bf36b9a18, State: Initialized, Role: FOLLOWER
I20260808 07:03:38.953969 16326 ts_tablet_manager.cc:1428] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:03:38.953990 16327 consensus_queue.cc:260] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.954041 16130 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
I20260808 07:03:38.954309 16327 ts_tablet_manager.cc:1428] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:03:38.954401 16307 heartbeater.cc:503] Master 127.24.181.126:40205 was elected leader, sending a full tablet report...
I20260808 07:03:38.986133 16332 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:38.986300 16332 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.986773 16332 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325), 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019)
I20260808 07:03:38.989814 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.986977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41325 (local address 127.24.181.66:45453)
0808 07:03:38.987038 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.987051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.987095 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.987263 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.987265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.987266 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:38.987351 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.987352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.988063 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.988064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.988897 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.988899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.989521 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.989523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.989604 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.989609 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.989662 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.989702 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:03:38.989817 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.987056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:45453 (local address 127.24.181.67:41325)
0808 07:03:38.987107 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.987109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.987120 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:03:38.987151 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.987152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.987163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.987235 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:38.987436 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.987978 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.988997 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.989436 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.989493 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.989500 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.989684 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.989725 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:03:38.989960 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.987045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:36773 (local address 127.24.181.65:44019)
0808 07:03:38.987112 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:38.987114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:38.987277 (+ 163us) server_negotiation.cc:407] Connection header received
0808 07:03:38.987312 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:38.987313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:38.987345 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:38.987413 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:38.987578 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.988169 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.989105 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.989561 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.989638 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.989722 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:38.989827 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:03:38.989876 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:03:38.990114 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:38.986977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.24.181.66:36773)
0808 07:03:38.987193 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:38.987204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:38.987261 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:38.987425 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:38.987426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:38.987427 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:38.987504 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.987505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.988252 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.988253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.989014 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:38.989016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:38.989665 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:38.989666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:38.989739 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:38.989743 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:03:38.989803 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:03:38.989854 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":21,"thread_start_us":118,"threads_started":1}
I20260808 07:03:38.990245 16245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8335a294df4db181969b136a425ad9" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be34d152b57243d4b74bf62bf36b9a18" is_pre_election: true
I20260808 07:03:38.990435 16245 raft_consensus.cc:2454] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 0.
I20260808 07:03:38.990460 15878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8335a294df4db181969b136a425ad9" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" is_pre_election: true
I20260808 07:03:38.990640 15878 raft_consensus.cc:2454] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 0.
I20260808 07:03:38.990660 15974 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18; no voters:
I20260808 07:03:38.990787 16332 raft_consensus.cc:2781] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:03:38.990870 16332 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:03:38.990904 16332 raft_consensus.cc:3037] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:38.992198 16332 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.992350 16332 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 election: Requested vote from peers be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325), 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019)
I20260808 07:03:38.992524 15878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8335a294df4db181969b136a425ad9" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
I20260808 07:03:38.992628 15878 raft_consensus.cc:3037] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:38.993265 16245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8335a294df4db181969b136a425ad9" candidate_uuid: "7c732d109216457fb73cc044399c939b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be34d152b57243d4b74bf62bf36b9a18"
I20260808 07:03:38.993352 16245 raft_consensus.cc:3037] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:38.993569 15878 raft_consensus.cc:2454] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 1.
I20260808 07:03:38.993773 15975 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8, 7c732d109216457fb73cc044399c939b; no voters:
I20260808 07:03:38.993875 16332 raft_consensus.cc:2781] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:38.994055 16332 raft_consensus.cc:686] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 1 LEADER]: Becoming Leader. State: Replica: 7c732d109216457fb73cc044399c939b, State: Running, Role: LEADER
I20260808 07:03:38.994159 16332 consensus_queue.cc:237] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:38.994364 16245 raft_consensus.cc:2454] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c732d109216457fb73cc044399c939b in term 1.
I20260808 07:03:38.995127 15677 catalog_manager.cc:5495] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c732d109216457fb73cc044399c939b (127.24.181.66). New cstate: current_term: 1 leader_uuid: "7c732d109216457fb73cc044399c939b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } health_report { overall_health: UNKNOWN } } }
I20260808 07:03:39.002622 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:39.000319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41750 (local address 127.24.181.66:42563)
0808 07:03:39.000364 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:39.000367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:39.000377 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:03:39.000451 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:39.000451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:39.000469 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:39.000556 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:39.000677 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.001135 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.001885 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.002087 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.002159 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:39.002337 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:39.002386 (+ 49us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 07:03:39.002531 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41750: token expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:03:39.002681 16320 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41750: token expired
I20260808 07:03:39.008443 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:39.003038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.24.181.126:40205)
0808 07:03:39.003076 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:39.003077 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:39.003087 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:03:39.003153 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:39.003154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:39.003160 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:39.003204 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:39.003649 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.004100 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.004873 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.005049 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.005102 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:39.006204 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:39.006205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:39.006205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:39.006211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:39.007586 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:39.007891 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:39.007891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:39.007892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:39.007936 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:39.008082 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:39.008083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:39.008083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:39.008166 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:39.008233 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:39.008323 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:03:39.008383 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:03:39.011379 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:39.009194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.24.181.66:42563)
0808 07:03:39.009259 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:39.009261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:39.009270 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:03:39.009325 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:39.009326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:39.009336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:39.009405 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:39.009515 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.009952 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.010625 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:39.010820 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:39.010895 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:39.011058 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:39.011148 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:39.011207 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:39.011280 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:03:39.011301 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:03:39.013159 16245 raft_consensus.cc:1261] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Refusing update from remote peer 7c732d109216457fb73cc044399c939b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:39.013172 15878 raft_consensus.cc:1261] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Refusing update from remote peer 7c732d109216457fb73cc044399c939b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:39.013523 16337 consensus_queue.cc:1035] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [LEADER]: Connected to new peer: Peer: permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:39.013660 16332 consensus_queue.cc:1035] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:39.016733 16340 mvcc.cc:205] Tried to move back new op lower bound from 7316163047475277824 to 7316163047401013248. Current Snapshot: MvccSnapshot[applied={T|T < 7316163047475277824}]
I20260808 07:03:39.018224 15676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:39.018321 15676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:39.019065 15676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Sending DeleteTablet for 3 replicas of tablet 4c8335a294df4db181969b136a425ad9
I20260808 07:03:39.019389 15852 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8335a294df4db181969b136a425ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:39 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:51844
I20260808 07:03:39.019411 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8335a294df4db181969b136a425ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:39 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40462
I20260808 07:03:39.019484 15955 tablet_replica.cc:330] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: stopping tablet replica
I20260808 07:03:39.019550 15955 raft_consensus.cc:2229] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:39.019605 15955 raft_consensus.cc:2258] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:39.019624 16035 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8335a294df4db181969b136a425ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:39 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:41738
I20260808 07:03:39.019852 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15735
I20260808 07:03:39.019903 16352 tablet_replica.cc:330] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: stopping tablet replica
I20260808 07:03:39.020004 16352 raft_consensus.cc:2229] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:03:39.020093 16352 raft_consensus.cc:2258] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:39.032991 16349 tablet_replica.cc:330] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: stopping tablet replica
I20260808 07:03:39.033187 16349 raft_consensus.cc:2229] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:39.033355 16349 raft_consensus.cc:2258] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:39.033987 16352 ts_tablet_manager.cc:1905] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:39.033987 16349 ts_tablet_manager.cc:1905] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:39.036015 16352 ts_tablet_manager.cc:1918] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:39.036113 16352 log.cc:1196] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/4c8335a294df4db181969b136a425ad9
I20260808 07:03:39.036430 16352 ts_tablet_manager.cc:1939] T 4c8335a294df4db181969b136a425ad9 P 7c732d109216457fb73cc044399c939b: Deleting consensus metadata
I20260808 07:03:39.036696 16349 ts_tablet_manager.cc:1918] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:39.036862 16349 log.cc:1196] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/4c8335a294df4db181969b136a425ad9
I20260808 07:03:39.036883 15659 catalog_manager.cc:4841] TS 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): tablet 4c8335a294df4db181969b136a425ad9 (table test-table [id=ef287680986443b09b8bee5020fd9bfe]) successfully deleted
I20260808 07:03:39.037230 16349 ts_tablet_manager.cc:1939] T 4c8335a294df4db181969b136a425ad9 P be34d152b57243d4b74bf62bf36b9a18: Deleting consensus metadata
I20260808 07:03:39.037670 15659 catalog_manager.cc:4841] TS be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): tablet 4c8335a294df4db181969b136a425ad9 (table test-table [id=ef287680986443b09b8bee5020fd9bfe]) successfully deleted
W20260808 07:03:39.038594 15660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:44019 (error 108)
I20260808 07:03:39.038947 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15949
I20260808 07:03:39.039083 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:39.038953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44019 (local address 127.0.0.1:51854)
0808 07:03:39.039003 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:39.039042 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:03:39.044241 15660 catalog_manager.cc:4584] TS 2d4e2c167b6d4647b52adbd075a33cd8 (127.24.181.65:44019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c8335a294df4db181969b136a425ad9: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44019: connect: Connection refused (error 111)
I20260808 07:03:39.052946 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 16133
I20260808 07:03:39.060457 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 15633
I20260808 07:03:59.068288 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--webserver_interface=127.24.181.126
--webserver_port=43431
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:59.150624 16355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:59.150830 16355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:59.150853 16355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:59.152007 16355 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:03:59.152048 16355 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:03:59.152062 16355 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:03:59.152073 16355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:03:59.152087 16355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:59.152100 16355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:59.152112 16355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:03:59.152168 16355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:03:59.153638 16355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:40205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=43431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:59.153894 16355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:59.154145 16355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:59.159060 16361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.159073 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.159145 16360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:59.159315 16355 server_base.cc:1034] running on GCE node
I20260808 07:03:59.159571 16355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:59.159894 16355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:59.161038 16355 hybrid_clock.cc:648] HybridClock initialized: now 1786172639161027 us; error 29 us; skew 500 ppm
Aug 08 07:03:59 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:59.166683 16355 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:03:59.167218 16355 webserver.cc:460] Webserver started at http://127.24.181.126:43431/ using document root <none> and password file <none>
I20260808 07:03:59.167474 16355 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:59.167546 16355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:59.168956 16355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:59.169700 16374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.169963 16355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:59.170079 16355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "3d7f0877cabf4d2fa7c62bae42bafeed"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "ac291cab52226c6477a7d4ad2af2729e"
server_key_iv: "d3a1d9bb1cf412b7db17c2ae6e8d59cd"
server_key_version: "encryptionkey@0"
I20260808 07:03:59.170452 16355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:59.178694 16355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:59.181056 16355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:59.181217 16355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:59.185238 16355 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:40205
I20260808 07:03:59.185315 16445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:40205 every 8 connection(s)
I20260808 07:03:59.185673 16355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:03:59.187232 16446 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:03:59.187896 16446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap starting.
I20260808 07:03:59.194109 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 16355
I20260808 07:03:59.194424 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:44019
--local_ip_for_outbound_sockets=127.24.181.65
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=46497
--webserver_interface=127.24.181.65
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:59.197901 16446 log.cc:824] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:59.200188 16446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:03:59.200520 16446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Bootstrap complete.
I20260808 07:03:59.202831 16446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:59.203145 16446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Initialized, Role: FOLLOWER
I20260808 07:03:59.203356 16446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:59.203442 16446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:03:59.203498 16446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:03:59.203547 16446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:03:59.204566 16446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:59.204689 16446 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7f0877cabf4d2fa7c62bae42bafeed; no voters:
I20260808 07:03:59.204916 16446 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:03:59.204952 16454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:03:59.205083 16454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [term 3 LEADER]: Becoming Leader. State: Replica: 3d7f0877cabf4d2fa7c62bae42bafeed, State: Running, Role: LEADER
I20260808 07:03:59.205185 16446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:03:59.205171 16454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } }
I20260808 07:03:59.205469 16454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7f0877cabf4d2fa7c62bae42bafeed. Latest consensus state: current_term: 3 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:59.205545 16454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:59.205744 16455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7f0877cabf4d2fa7c62bae42bafeed" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 40205 } } }
I20260808 07:03:59.205883 16455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:59.205791 16460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:03:59.206969 16460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:03:59.207130 16460 catalog_manager.cc:1259] Loaded cluster ID: 712a4ce1c3564e1289de104ddfc1060b
I20260808 07:03:59.207373 16460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:03:59.208163 16460 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:03:59.208376 16460 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Loaded TSK: 0
I20260808 07:03:59.208438 16460 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d7f0877cabf4d2fa7c62bae42bafeed: Loaded TSK: 1
F20260808 07:03:59.208653 16460 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9de44d49ad google::LogMessage::Fail() at ??:0
@ 0x7f9de44d788a google::LogMessage::SendToLog() at ??:0
@ 0x7f9de44d44de google::LogMessage::Flush() at ??:0
@ 0x7f9de44d5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9de6c41104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9de6c3ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9dec42e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9dec42e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9dec42e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9dec3f4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f9dec3fead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9de63a20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9de6397954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9de71fb6db start_thread at ??:0
@ 0x7f9de2fd261f clone at ??:0
W20260808 07:03:59.296675 16448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:59.296902 16448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:59.296942 16448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:59.298094 16448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:59.298168 16448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:59.298216 16448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:03:59.299754 16448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:44019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=46497
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:59.300020 16448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:59.300313 16448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:03:59.305528 16448 server_base.cc:1034] running on GCE node
W20260808 07:03:59.305395 16498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.305399 16499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.305399 16501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:59.305878 16448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:59.306152 16448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:59.307330 16448 hybrid_clock.cc:648] HybridClock initialized: now 1786172639307313 us; error 34 us; skew 500 ppm
Aug 08 07:03:59 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:59.310415 16448 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:03:59.310973 16448 webserver.cc:460] Webserver started at http://127.24.181.65:46497/ using document root <none> and password file <none>
I20260808 07:03:59.311272 16448 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:59.311347 16448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:59.313341 16448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:03:59.314306 16512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.314504 16448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:59.314587 16448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "2d4e2c167b6d4647b52adbd075a33cd8"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "b0310a32cfd4f84e3954fc83b7c42001"
server_key_iv: "f37018dc7508e54896cb28541850110b"
server_key_version: "encryptionkey@0"
I20260808 07:03:59.314884 16448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:59.325717 16448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:59.328111 16448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:59.328296 16448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:59.328516 16448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:59.328935 16520 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:03:59.331123 16448 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:03:59.331209 16448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:03:59.331259 16448 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:03:59.331907 16448 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:03:59.331966 16448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:59.332047 16520 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap starting.
I20260808 07:03:59.339520 16448 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:44019
I20260808 07:03:59.339668 16677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:44019 every 8 connection(s)
I20260808 07:03:59.339938 16448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:03:59.340775 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 16448
I20260808 07:03:59.341074 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:42563
--local_ip_for_outbound_sockets=127.24.181.66
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=34827
--webserver_interface=127.24.181.66
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:03:59.346370 16520 log.cc:824] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:59.349397 16520 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:03:59.349793 16520 tablet_bootstrap.cc:492] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Bootstrap complete.
I20260808 07:03:59.351372 16520 ts_tablet_manager.cc:1397] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.016s
I20260808 07:03:59.352967 16520 raft_consensus.cc:348] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:59.353323 16520 raft_consensus.cc:729] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e2c167b6d4647b52adbd075a33cd8, State: Initialized, Role: FOLLOWER
I20260808 07:03:59.353477 16520 consensus_queue.cc:260] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:03:59.353783 16520 ts_tablet_manager.cc:1428] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260808 07:03:59.425534 16682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:03:59.340722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:46183)
0808 07:03:59.340938 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.340985 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:59.341795 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:59.425318 (+ 83523us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
W20260808 07:03:59.425814 16678 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:40205 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:40205: Client connection negotiation failed: client connection to 127.24.181.126:40205: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:03:59.426254 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.426090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:57727)
0808 07:03:59.426148 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.426230 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:03:59.426705 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.426563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:59589)
0808 07:03:59.426648 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.426682 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 07:03:59.426868 16678 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:03:59.426973 16683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:59.427153 16683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:59.427177 16683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:59.428295 16683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:59.428332 16683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:59.428359 16683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:03:59.429736 16683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:42563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=34827
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:59.429913 16683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:59.430083 16683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:59.435210 16696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.435309 16693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.435225 16692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:59.435312 16683 server_base.cc:1034] running on GCE node
I20260808 07:03:59.435591 16683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:59.435848 16683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:59.437016 16683 hybrid_clock.cc:648] HybridClock initialized: now 1786172639436997 us; error 28 us; skew 500 ppm
Aug 08 07:03:59 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:59.439662 16683 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:03:59.440099 16683 webserver.cc:460] Webserver started at http://127.24.181.66:34827/ using document root <none> and password file <none>
I20260808 07:03:59.440333 16683 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:59.440403 16683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:59.441771 16683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:59.442384 16706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.442560 16683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:03:59.442664 16683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "7c732d109216457fb73cc044399c939b"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "a5985d9c675870b32790dc2cf54f8334"
server_key_iv: "98a925a5ef41cac2fcdb36f0c9919895"
server_key_version: "encryptionkey@0"
I20260808 07:03:59.443051 16683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:59.453287 16683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:59.455575 16683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:59.455734 16683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:59.455960 16683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:59.456264 16683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:59.456317 16683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.456358 16683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:59.456398 16683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.462380 16683 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:42563
I20260808 07:03:59.462455 16861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:42563 every 8 connection(s)
I20260808 07:03:59.462805 16683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:03:59.463924 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.463546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:35567)
0808 07:03:59.463696 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.463765 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260808 07:03:59.464339 16862 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:40205 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:40205: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
I20260808 07:03:59.464624 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.464542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:43173)
0808 07:03:59.464584 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.464609 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 07:03:59.464938 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.464841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:59919)
0808 07:03:59.464889 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.464915 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:03:59.465044 16862 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:03:59.466738 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 16683
I20260808 07:03:59.467054 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:41325
--local_ip_for_outbound_sockets=127.24.181.67
--tserver_master_addrs=127.24.181.126:40205
--webserver_port=34355
--webserver_interface=127.24.181.67
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:59.552878 16866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:59.553112 16866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:59.553151 16866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:59.554273 16866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:59.554344 16866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:59.554387 16866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:03:59.555892 16866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:41325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=34355
--tserver_master_addrs=127.24.181.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:03:59.556155 16866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:59.556386 16866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:59.561461 16879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:59.561595 16866 server_base.cc:1034] running on GCE node
W20260808 07:03:59.561444 16876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:59.561439 16877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:59.561957 16866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:59.562217 16866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:59.563396 16866 hybrid_clock.cc:648] HybridClock initialized: now 1786172639563384 us; error 39 us; skew 500 ppm
Aug 08 07:03:59 dist-test-slave-nfb5 krb5kdc[14868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:59.565829 16866 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:03:59.566406 16866 webserver.cc:460] Webserver started at http://127.24.181.67:34355/ using document root <none> and password file <none>
I20260808 07:03:59.566666 16866 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:59.566736 16866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:59.568104 16866 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:59.568717 16892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.568899 16866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:03:59.568997 16866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "be34d152b57243d4b74bf62bf36b9a18"
format_stamp: "Formatted at 2026-08-08 07:03:17 on dist-test-slave-nfb5"
server_key: "7972fec3ad0a7bbc11d831416490e938"
server_key_iv: "d9e373337f7474fb347673589dfcd351"
server_key_version: "encryptionkey@0"
I20260808 07:03:59.569365 16866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:59.594671 16866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:59.596794 16866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:59.596966 16866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:59.597219 16866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:59.597534 16866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:59.597590 16866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.597633 16866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:59.597662 16866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:59.603976 16866 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:41325
I20260808 07:03:59.604099 17049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:41325 every 8 connection(s)
I20260808 07:03:59.604466 16866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:59.605552 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.605185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:40753)
0808 07:03:59.605344 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.605415 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260808 07:03:59.605988 17050 heartbeater.cc:644] Failed to heartbeat to 127.24.181.126:40205 (0 consecutive failures): Network error: Failed to ping master at 127.24.181.126:40205: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
I20260808 07:03:59.606310 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.606207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:53657)
0808 07:03:59.606251 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.606290 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 07:03:59.606602 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:59.606530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:49009)
0808 07:03:59.606566 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:59.606588 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:03:59.606729 17050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:03:59.612771 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 16866
I20260808 07:04:00.427577 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.427307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:46347)
0808 07:04:00.427467 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:00.427510 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260808 07:04:00.465586 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.465366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:36037)
0808 07:04:00.465501 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:00.465535 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260808 07:04:00.607374 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.607134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:48051)
0808 07:04:00.607252 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:00.607289 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260808 07:04:00.701287 17061 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:00.701431 17061 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:00.701761 17061 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
Aug 08 07:04:00 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.66@KRBTEST.COM
Aug 08 07:04:00 dist-test-slave-nfb5 krb5kdc[14868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.67@KRBTEST.COM
I20260808 07:04:00.713286 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.702132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:52107 (local address 127.24.181.66:42563)
0808 07:04:00.702322 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:00.702331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:00.702359 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:04:00.702845 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:00.702849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:00.702952 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:00.703101 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:00.703952 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.704787 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.705790 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.706217 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.706348 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:00.708910 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:00.708914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:00.708914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:00.708924 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:00.711064 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711756 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:00.711760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:00.711760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:00.711873 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:00.712194 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:00.712195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:00.712195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:00.712528 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:00.712654 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:00.712997 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 07:04:00.713159 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260808 07:04:00.713274 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.702043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:41325 (local address 127.24.181.65:38515)
0808 07:04:00.702183 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:00.702191 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:00.702228 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:00.703301 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:00.703302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:00.703552 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:00.703798 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:00.703804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.704905 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.704906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:00.705716 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:00.705719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.706325 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.706326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:00.706460 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:00.707248 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:00.707253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:00.709179 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:00.711272 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:00.711273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:00.711650 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:00.711961 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:00.711963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:00.712113 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:00.712632 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:00.712639 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:00.712786 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 07:04:00.712902 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:04:00.713098 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":61,"thread_start_us":58,"threads_started":1}
I20260808 07:04:00.713810 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.702160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:38515 (local address 127.24.181.67:41325)
0808 07:04:00.702298 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:00.702316 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:00.702347 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:04:00.702888 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:00.702890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:00.702985 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:00.703153 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:00.703925 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.704792 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.705823 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.706236 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.706365 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:00.709286 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:00.709291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:00.709291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:00.709308 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:00.711172 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711752 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:00.711755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:00.711756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:00.711876 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:00.712191 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:00.712192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:00.712192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:00.712519 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:00.712599 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:00.713513 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 07:04:00.713654 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260808 07:04:00.713354 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:00.701928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42563 (local address 127.24.181.65:52107)
0808 07:04:00.701993 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:00.702007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:00.702083 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:00.703139 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:00.703142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:00.703589 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:00.703798 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:00.703804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.704910 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.704911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:00.705692 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:00.705695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:00.706318 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:00.706319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:00.706459 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:00.707157 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:00.707168 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:00.708786 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:00.711147 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711169 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:00.711171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:00.711649 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:00.711990 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:00.711991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:00.711992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:00.712110 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:00.712638 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:00.712639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:00.712809 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 07:04:00.712909 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:04:00.713186 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":197}
W20260808 07:04:00.715199 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:00.715309 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:00.715364 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:00.715482 17061 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:01.428285 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:01.428017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:45883)
0808 07:04:01.428188 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:01.428235 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
I20260808 07:04:01.466217 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:01.466011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:59525)
0808 07:04:01.466141 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:01.466179 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260808 07:04:01.608104 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:01.607890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:32825)
0808 07:04:01.608019 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:01.608061 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260808 07:04:02.429055 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:02.428823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:45333)
0808 07:04:02.428969 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:02.429009 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260808 07:04:02.466882 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:02.466680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:49603)
0808 07:04:02.466808 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:02.466841 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260808 07:04:02.608856 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:02.608556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:59443)
0808 07:04:02.608762 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:02.608800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":59,"threads_started":1}
I20260808 07:04:02.765482 17081 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:02.765585 17081 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:02.765865 17081 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
W20260808 07:04:02.766360 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:02.766462 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:02.766500 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:02.766595 17081 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:03.429737 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:03.429517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:53689)
0808 07:04:03.429645 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:03.429689 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260808 07:04:03.467567 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:03.467351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:54427)
0808 07:04:03.467492 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:03.467528 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 07:04:03.609496 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:03.609288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:48583)
0808 07:04:03.609427 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:03.609461 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260808 07:04:04.430546 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:04.430316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:43871)
0808 07:04:04.430455 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:04.430500 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260808 07:04:04.468267 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:04.468052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:50045)
0808 07:04:04.468191 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:04.468225 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 07:04:04.610152 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:04.609942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:60121)
0808 07:04:04.610073 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:04.610113 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 07:04:05.085352 17090 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:05.085464 17090 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:05.085634 17090 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
W20260808 07:04:05.086043 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:05.086138 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:05.086172 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:05.086274 17090 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:05.431484 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:05.431218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:42717)
0808 07:04:05.431390 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:05.431431 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260808 07:04:05.468992 17093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:05.468751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:41251)
0808 07:04:05.468909 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:05.468947 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260808 07:04:05.610904 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:05.610697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:46593)
0808 07:04:05.610826 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:05.610862 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260808 07:04:06.432397 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:06.432153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:39747)
0808 07:04:06.432286 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:06.432333 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260808 07:04:06.469758 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:06.469501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:58245)
0808 07:04:06.469644 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:06.469702 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260808 07:04:06.611691 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:06.611445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:55923)
0808 07:04:06.611601 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:06.611640 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260808 07:04:07.433148 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:07.432922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:44963)
0808 07:04:07.433059 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:07.433104 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260808 07:04:07.470517 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:07.470286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:41975)
0808 07:04:07.470431 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:07.470472 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260808 07:04:07.612450 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:07.612214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:58885)
0808 07:04:07.612371 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:07.612407 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260808 07:04:08.373194 17105 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:08.373319 17105 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:08.373509 17105 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
W20260808 07:04:08.373930 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:08.374009 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:08.374027 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:08.374127 17105 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:08.434003 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:08.433749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:34907)
0808 07:04:08.433909 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:08.433953 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260808 07:04:08.471289 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:08.471048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:33003)
0808 07:04:08.471180 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:08.471223 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260808 07:04:08.613319 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:08.613054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:36993)
0808 07:04:08.613226 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:08.613269 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260808 07:04:09.434966 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:09.434600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:55353)
0808 07:04:09.434837 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:09.434878 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":115,"threads_started":1}
I20260808 07:04:09.471949 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:09.471731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:51287)
0808 07:04:09.471866 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:09.471904 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260808 07:04:09.613981 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:09.613762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:45385)
0808 07:04:09.613899 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:09.613936 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260808 07:04:10.435767 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:10.435475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:56067)
0808 07:04:10.435628 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:10.435689 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260808 07:04:10.472678 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:10.472398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:41331)
0808 07:04:10.472588 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:10.472629 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260808 07:04:10.614696 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:10.614476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:52475)
0808 07:04:10.614619 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:10.614650 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260808 07:04:10.831842 17116 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:10.831935 17116 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:10.832095 17116 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
W20260808 07:04:10.832526 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:10.832592 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:10.832607 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:10.832727 17116 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:11.436753 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:11.436485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:42791)
0808 07:04:11.436624 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:11.436666 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260808 07:04:11.473454 17118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:11.473239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:52721)
0808 07:04:11.473374 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:11.473410 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260808 07:04:11.615409 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:11.615180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:52925)
0808 07:04:11.615319 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:11.615362 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260808 07:04:12.437513 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:12.437266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:37469)
0808 07:04:12.437422 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:12.437464 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260808 07:04:12.474177 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:12.473957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:37727)
0808 07:04:12.474085 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:12.474128 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260808 07:04:12.616104 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:12.615896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:42925)
0808 07:04:12.616017 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:12.616055 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260808 07:04:13.438357 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:13.438077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:38039)
0808 07:04:13.438259 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:13.438306 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260808 07:04:13.475080 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:13.474785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:43281)
0808 07:04:13.474992 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:13.475032 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":68,"threads_started":1}
I20260808 07:04:13.616858 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:13.616600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:53949)
0808 07:04:13.616755 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:13.616798 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260808 07:04:14.439170 17131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:14.438897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.65:44167)
0808 07:04:14.439086 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:14.439122 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260808 07:04:14.475885 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:14.475638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.66:49709)
0808 07:04:14.475787 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:14.475832 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260808 07:04:14.595199 17133 raft_consensus.cc:482] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:14.595307 17133 raft_consensus.cc:504] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c732d109216457fb73cc044399c939b" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42563 } } peers { permanent_uuid: "be34d152b57243d4b74bf62bf36b9a18" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 41325 } } peers { permanent_uuid: "2d4e2c167b6d4647b52adbd075a33cd8" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44019 } }
I20260808 07:04:14.595582 17133 leader_election.cc:288] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c732d109216457fb73cc044399c939b (127.24.181.66:42563), be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325)
W20260808 07:04:14.596086 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c732d109216457fb73cc044399c939b (127.24.181.66:42563): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
W20260808 07:04:14.596176 16513 leader_election.cc:341] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be34d152b57243d4b74bf62bf36b9a18 (127.24.181.67:41325): Not found: Tablet not found: 4c8335a294df4db181969b136a425ad9
I20260808 07:04:14.596248 16513 leader_election.cc:302] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d4e2c167b6d4647b52adbd075a33cd8; no voters: 7c732d109216457fb73cc044399c939b, be34d152b57243d4b74bf62bf36b9a18
I20260808 07:04:14.596356 17133 raft_consensus.cc:2726] T 4c8335a294df4db181969b136a425ad9 P 2d4e2c167b6d4647b52adbd075a33cd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:04:14.616797 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 16448
I20260808 07:04:14.624081 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:14.623292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:40205 (local address 127.24.181.67:56599)
0808 07:04:14.623960 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:14.623999 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.126:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":586,"threads_started":1}
I20260808 07:04:14.632122 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 16683
I20260808 07:04:14.641230 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 16866
2026-08-08T07:04:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58649 ms)
[----------] 30 tests from SecurityITest (247052 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:14 dist-test-slave-nfb5 krb5kdc[17138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:14 dist-test-slave-nfb5 krb5kdc[17138](info): set up 2 sockets
Aug 08 07:04:14 dist-test-slave-nfb5 krb5kdc[17138](info): commencing operation
krb5kdc: starting...
W20260808 07:04:15.878789 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:15 dist-test-slave-nfb5 krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:16.006834 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44123
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:16.090004 17157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:16.090209 17157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:16.090231 17157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:16.091409 17157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:16.091454 17157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:16.091467 17157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:16.091480 17157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:16.091524 17157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:16.093058 17157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:16.093317 17157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:16.093571 17157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:16.098750 17157 server_base.cc:1034] running on GCE node
W20260808 07:04:16.098807 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.098807 17164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.098750 17167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.099337 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:16.099663 17157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:16.100859 17157 hybrid_clock.cc:648] HybridClock initialized: now 1786172656100844 us; error 45 us; skew 500 ppm
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:16.103607 17157 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:16.103998 17157 webserver.cc:460] Webserver started at http://127.24.181.126:46747/ using document root <none> and password file <none>
I20260808 07:04:16.104212 17157 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:16.104255 17157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:16.104341 17157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:16.105346 17157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "f70787b87e254ec884de8645089ef9b4"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "15f3ddd92d31e44e0c72986a4252d77e"
server_key_iv: "98680bce1252ba5cb6c9b92eb776f7f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.105670 17157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "f70787b87e254ec884de8645089ef9b4"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "15f3ddd92d31e44e0c72986a4252d77e"
server_key_iv: "98680bce1252ba5cb6c9b92eb776f7f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.106993 17157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:16.107755 17177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.107920 17157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:16.108011 17157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "f70787b87e254ec884de8645089ef9b4"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "15f3ddd92d31e44e0c72986a4252d77e"
server_key_iv: "98680bce1252ba5cb6c9b92eb776f7f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.108068 17157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:16.124758 17157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:16.127180 17157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:16.127295 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:16.132138 17157 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44123
I20260808 07:04:16.132181 17249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44123 every 8 connection(s)
I20260808 07:04:16.132612 17157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:16.133585 17250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:16.136473 17250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: Bootstrap starting.
I20260808 07:04:16.137006 17250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:16.137243 17250 log.cc:824] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:16.137952 17250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: No bootstrap required, opened a new log
I20260808 07:04:16.139694 17250 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } }
I20260808 07:04:16.139843 17250 raft_consensus.cc:374] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:16.139868 17250 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70787b87e254ec884de8645089ef9b4, State: Initialized, Role: FOLLOWER
I20260808 07:04:16.139967 17250 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } }
I20260808 07:04:16.140049 17250 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:16.140089 17250 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:16.140139 17250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:16.140677 17250 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } }
I20260808 07:04:16.140810 17250 leader_election.cc:302] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70787b87e254ec884de8645089ef9b4; no voters:
I20260808 07:04:16.140992 17250 leader_election.cc:288] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:16.141036 17255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:16.141203 17255 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [term 1 LEADER]: Becoming Leader. State: Replica: f70787b87e254ec884de8645089ef9b4, State: Running, Role: LEADER
I20260808 07:04:16.141285 17250 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:16.141287 17255 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } }
I20260808 07:04:16.141752 17256 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f70787b87e254ec884de8645089ef9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } } }
I20260808 07:04:16.141845 17256 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:16.141993 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70787b87e254ec884de8645089ef9b4. Latest consensus state: current_term: 1 leader_uuid: "f70787b87e254ec884de8645089ef9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70787b87e254ec884de8645089ef9b4" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44123 } } }
I20260808 07:04:16.142062 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:16.142665 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 17157
I20260808 07:04:16.142808 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:16.143052 25301 external_mini_cluster.cc:1428] Setting key 3fd9f7f3071bce642658b2406878fd54
W20260808 07:04:16.143304 17278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:16.143373 17278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:16.143419 17268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:16.143563 17268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:16.145208 17268 catalog_manager.cc:1347] Generated new cluster ID: 109452af44254b95a7848ef96db0aa9d
I20260808 07:04:16.145262 17268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:16.155797 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.144175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.24.181.126:44123)
0808 07:04:16.144356 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.144365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.144495 (+ 130us) server_negotiation.cc:407] Connection header received
0808 07:04:16.144816 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.144824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.144883 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.145032 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:16.145823 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.147350 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.148386 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.149016 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.149141 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.152033 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:16.152038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:16.152038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:16.152050 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:16.154086 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.154525 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.154529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.154529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.154632 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.154851 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.154852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.154852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.155165 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:16.155321 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.155438 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.155570 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260808 07:04:16.170071 17268 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:16.170604 17268 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:16.175835 17268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: Generated new TSK 0
I20260808 07:04:16.176024 17268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:04:16.214682 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:16.319696 17294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:16.319895 17294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:16.319918 17294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:16.321151 17294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:16.321194 17294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:16.321224 17294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:16.322594 17294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:16.322778 17294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:16.322989 17294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:16.327939 17303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.327993 17301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.328017 17294 server_base.cc:1034] running on GCE node
W20260808 07:04:16.327951 17300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.328429 17294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:16.328620 17294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:16.329793 17294 hybrid_clock.cc:648] HybridClock initialized: now 1786172656329767 us; error 39 us; skew 500 ppm
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:16.332494 17294 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:04:16.333029 17294 webserver.cc:460] Webserver started at http://127.24.181.65:39469/ using document root <none> and password file <none>
I20260808 07:04:16.333280 17294 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:16.333328 17294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:16.333483 17294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:16.334515 17294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "dc6a058dd2944a97922c63bc66f3a385"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2e30035117442a64e1a0b20b61e64516"
server_key_iv: "7a797dedca8c0978cf542f3a8cecaa59"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.334898 17294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "dc6a058dd2944a97922c63bc66f3a385"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2e30035117442a64e1a0b20b61e64516"
server_key_iv: "7a797dedca8c0978cf542f3a8cecaa59"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.336503 17294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:16.337208 17314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.337396 17294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:16.337496 17294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "dc6a058dd2944a97922c63bc66f3a385"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2e30035117442a64e1a0b20b61e64516"
server_key_iv: "7a797dedca8c0978cf542f3a8cecaa59"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.337584 17294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:16.354051 17294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:16.356098 17294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:16.356276 17294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:16.356511 17294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:16.356838 17294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:16.356899 17294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.356940 17294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:16.356967 17294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.363025 17294 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:34861
I20260808 07:04:16.363076 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:34861 every 8 connection(s)
I20260808 07:04:16.363410 17294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:16.371336 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 17294
I20260808 07:04:16.371467 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:16.371729 25301 external_mini_cluster.cc:1428] Setting key 041a297b3d6e004ecb8a98214bcc6f3c
I20260808 07:04:16.372859 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.364199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:58893 (local address 127.24.181.126:44123)
0808 07:04:16.364288 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.364291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.364457 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:04:16.364939 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.364940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.364955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.365010 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:16.365960 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.366458 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.367422 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.367622 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.367673 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.369686 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:16.369688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:16.369688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:16.369695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:16.371211 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.371834 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.371836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.371836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.371902 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.372197 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.372198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.372198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.372293 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:16.372385 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.372665 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.372761 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:04:16.373076 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.364136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44123 (local address 127.24.181.65:58893)
0808 07:04:16.364329 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.364364 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.364852 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.365062 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.365065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.365672 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:16.365884 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.365888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.366542 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.366543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.367349 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.367351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.367699 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.367700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.367814 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.368258 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:16.368266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:16.369590 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:16.371311 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.371321 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.371323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.371652 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.371963 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.371964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.371964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.372131 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.372376 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:16.372385 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:16.372525 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.372641 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.372811 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
I20260808 07:04:16.373561 17491 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44123
I20260808 07:04:16.373652 17491 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:16.373852 17491 heartbeater.cc:511] Master 127.24.181.126:44123 requested a full tablet report, sending...
I20260808 07:04:16.374387 17203 ts_manager.cc:194] Registered new tserver with Master: dc6a058dd2944a97922c63bc66f3a385 (127.24.181.65:34861)
I20260808 07:04:16.375046 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:58893
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:04:16.398447 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:16.485221 17501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:16.485486 17501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:16.485530 17501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:16.486846 17501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:16.486931 17501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:16.487003 17501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:16.488655 17501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:16.488969 17501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:16.489231 17501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:16.495044 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.495030 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.495010 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.495255 17501 server_base.cc:1034] running on GCE node
I20260808 07:04:16.495478 17501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:16.495750 17501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:16.496970 17501 hybrid_clock.cc:648] HybridClock initialized: now 1786172656496953 us; error 29 us; skew 500 ppm
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:16.499575 17501 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:04:16.500056 17501 webserver.cc:460] Webserver started at http://127.24.181.66:43273/ using document root <none> and password file <none>
I20260808 07:04:16.500288 17501 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:16.500360 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:16.500516 17501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:16.501580 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "7d6d7f9a37234173958b4ff5d1670028"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "4172c7932bc26a6752cf69fb05c2e1f5"
server_key_iv: "bad9ea44288c98038b5cd32be88263a0"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.501953 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "7d6d7f9a37234173958b4ff5d1670028"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "4172c7932bc26a6752cf69fb05c2e1f5"
server_key_iv: "bad9ea44288c98038b5cd32be88263a0"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.503398 17501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:16.504150 17528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.504333 17501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:16.504434 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "7d6d7f9a37234173958b4ff5d1670028"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "4172c7932bc26a6752cf69fb05c2e1f5"
server_key_iv: "bad9ea44288c98038b5cd32be88263a0"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.504534 17501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:16.523355 17501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:16.525367 17501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:16.525534 17501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:16.525782 17501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:16.526109 17501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:16.526170 17501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.526212 17501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:16.526238 17501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.532179 17501 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:45077
I20260808 07:04:16.532260 17698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:45077 every 8 connection(s)
I20260808 07:04:16.532590 17501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:16.534176 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 17501
I20260808 07:04:16.534328 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:16.534549 25301 external_mini_cluster.cc:1428] Setting key 6b58edb901e8404d78e543d12fe8cbdf
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:16.544275 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.533473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:53657 (local address 127.24.181.126:44123)
0808 07:04:16.533570 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.533573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.533813 (+ 240us) server_negotiation.cc:407] Connection header received
0808 07:04:16.534763 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.534765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.534781 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.534859 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:16.536393 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.536878 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.538090 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.538371 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.538430 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.541150 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:16.541152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:16.541152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:16.541160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:16.542846 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.543289 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.543290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.543290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.543358 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.543614 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.543615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.543616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.543740 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:16.543815 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.544091 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.544180 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:04:16.544483 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.533376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44123 (local address 127.24.181.66:53657)
0808 07:04:16.533611 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.533709 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.534560 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.535054 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.535057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.535978 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:16.536283 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.536290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.536981 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.536982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.537991 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.537993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.538470 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.538471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.538636 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.539216 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:16.539226 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:16.541049 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:16.542952 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.542961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.542963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.543187 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.543431 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.543432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.543433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.543539 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.543817 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:16.543826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:16.543958 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.544069 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.544229 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
I20260808 07:04:16.544974 17701 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44123
I20260808 07:04:16.545059 17701 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:16.545257 17701 heartbeater.cc:511] Master 127.24.181.126:44123 requested a full tablet report, sending...
I20260808 07:04:16.545662 17203 ts_manager.cc:194] Registered new tserver with Master: 7d6d7f9a37234173958b4ff5d1670028 (127.24.181.66:45077)
I20260808 07:04:16.546167 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:53657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:04:16.561975 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:16.645255 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:16.645496 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:16.645540 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:16.646704 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:16.646777 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:16.646824 17710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:16.648377 17710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:16.648658 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:16.648903 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:16.654109 17719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.654153 17710 server_base.cc:1034] running on GCE node
W20260808 07:04:16.654129 17718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:16.654106 17721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:16.654525 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:16.654783 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:16.655956 17710 hybrid_clock.cc:648] HybridClock initialized: now 1786172656655951 us; error 45 us; skew 500 ppm
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:16.658622 17710 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:04:16.659209 17710 webserver.cc:460] Webserver started at http://127.24.181.67:41073/ using document root <none> and password file <none>
I20260808 07:04:16.659474 17710 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:16.659550 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:16.659706 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:16.660785 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "db72ebb7c719497db36b26ddb70cd021"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2dd8f099ba772cdad4a3175b40ff0b53"
server_key_iv: "8c45a2223ab20d22f899e4e58878a2f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.661155 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "db72ebb7c719497db36b26ddb70cd021"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2dd8f099ba772cdad4a3175b40ff0b53"
server_key_iv: "8c45a2223ab20d22f899e4e58878a2f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.662508 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:16.663303 17734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.663493 17710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.663587 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "db72ebb7c719497db36b26ddb70cd021"
format_stamp: "Formatted at 2026-08-08 07:04:16 on dist-test-slave-nfb5"
server_key: "2dd8f099ba772cdad4a3175b40ff0b53"
server_key_iv: "8c45a2223ab20d22f899e4e58878a2f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:16.663671 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:16.677512 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:16.679592 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:16.679759 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:16.680001 17710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:16.680315 17710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:16.680352 17710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.680375 17710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:16.680388 17710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:16.686255 17710 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:42555
I20260808 07:04:16.686316 17896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:42555 every 8 connection(s)
I20260808 07:04:16.686661 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:16.687098 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 17710
I20260808 07:04:16.687225 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:16.687445 25301 external_mini_cluster.cc:1428] Setting key 07f2dab3905d06f0fe893d716ad52179
Aug 08 07:04:16 dist-test-slave-nfb5 krb5kdc[17138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172656, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:16.696853 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.687682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:49627 (local address 127.24.181.126:44123)
0808 07:04:16.687731 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.687734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.687937 (+ 203us) server_negotiation.cc:407] Connection header received
0808 07:04:16.688775 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.688777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.688791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.688838 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:16.689911 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.690399 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.691548 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.691753 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.691812 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.694144 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:16.694145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:16.694146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:16.694153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:16.695649 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.696039 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.696040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.696040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.696093 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.696321 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.696321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.696322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.696423 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:16.696474 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.696723 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.696786 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:04:16.697003 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.687511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44123 (local address 127.24.181.67:49627)
0808 07:04:16.687816 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.687861 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.688536 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.688957 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.688961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.689609 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:16.689825 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.689830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.690488 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.690489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.691468 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.691470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.691843 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.691844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.691955 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.692455 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:16.692464 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:16.694049 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:16.695727 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.695734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.695735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.695965 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.696159 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:16.696160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:16.696160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:16.696263 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:16.696480 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:16.696485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:16.696598 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.696680 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.696802 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260808 07:04:16.697548 17897 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44123
I20260808 07:04:16.697665 17897 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:16.697849 17897 heartbeater.cc:511] Master 127.24.181.126:44123 requested a full tablet report, sending...
I20260808 07:04:16.698256 17203 ts_manager.cc:194] Registered new tserver with Master: db72ebb7c719497db36b26ddb70cd021 (127.24.181.67:42555)
I20260808 07:04:16.698640 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:49627
I20260808 07:04:16.699645 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:16.706573 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.700958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.24.181.126:44123)
0808 07:04:16.701002 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.701004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.701128 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:04:16.701171 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.701172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.701179 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.701232 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:16.701748 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.702183 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.702844 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.703082 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.703134 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.704177 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:16.704178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:16.704179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:16.704185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:16.705626 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.705931 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.705932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.705932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.705981 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:16.706140 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:16.706141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:16.706141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:16.706268 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:16.706340 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.706435 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.706488 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:04:16.707990 17203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:16.709656 17203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:16.717321 17913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42555 (local address 127.0.0.1:54270)
0808 07:04:16.712346 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.712389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.712457 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.712933 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.712936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.712939 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:16.713019 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.713020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.714436 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.714437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.716071 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.716075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.716985 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.716986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.717075 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.717089 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.717180 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.717235 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":69,"thread_start_us":60,"threads_started":1}
I20260808 07:04:16.717799 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.24.181.67:42555)
0808 07:04:16.712666 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.712670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.712682 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:16.712721 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.712724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.712741 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.713105 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:16.713164 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.714278 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.716180 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.716897 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.716952 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.716991 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.717674 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.717716 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260808 07:04:16.717885 17915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:45077 (local address 127.0.0.1:57230)
0808 07:04:16.712824 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.712833 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.712871 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.714647 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.714649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.714651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:16.714704 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.714706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.715706 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.715707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.716639 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.716642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.717626 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.717627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.717696 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.717701 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.717767 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.717811 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":79,"threads_started":1}
I20260808 07:04:16.717922 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.24.181.66:45077)
0808 07:04:16.714486 (+ 1818us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.714490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.714498 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:16.714530 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.714532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.714544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.714636 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:16.714762 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.715591 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.716753 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.717511 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.717621 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.717692 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.717787 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.717829 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1770,"thread_start_us":138,"threads_started":1}
I20260808 07:04:16.718339 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34861 (local address 127.0.0.1:45492)
0808 07:04:16.712669 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.712680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.712743 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.712968 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.712969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.712970 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:16.713020 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.713021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.714438 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.714438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.716072 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.716075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.718085 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.718086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.718161 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.718165 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.718225 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.718276 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":53,"thread_start_us":100,"threads_started":1}
I20260808 07:04:16.718601 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.712277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.24.181.65:34861)
0808 07:04:16.712497 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.712502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.712806 (+ 304us) server_negotiation.cc:407] Connection header received
0808 07:04:16.712858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.712860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.712876 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.712974 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:16.713164 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.714278 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.717022 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.717915 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.717993 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.718048 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.718447 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.718490 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":110,"threads_started":1}
I20260808 07:04:16.718626 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 663f84aba2914117a35895f7c3bfbce6 (DEFAULT_TABLE table=test-table [id=58c0cb962288418388ed611377ba87a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:16.719121 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663f84aba2914117a35895f7c3bfbce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:16.719319 17393 tablet_service.cc:1467] Processing CreateTablet for tablet 663f84aba2914117a35895f7c3bfbce6 (DEFAULT_TABLE table=test-table [id=58c0cb962288418388ed611377ba87a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:16.719734 17393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663f84aba2914117a35895f7c3bfbce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:16.723160 17924 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Bootstrap starting.
I20260808 07:04:16.723444 17926 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Bootstrap starting.
I20260808 07:04:16.723577 17601 tablet_service.cc:1467] Processing CreateTablet for tablet 663f84aba2914117a35895f7c3bfbce6 (DEFAULT_TABLE table=test-table [id=58c0cb962288418388ed611377ba87a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:16.723932 17924 tablet_bootstrap.cc:654] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:16.723932 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663f84aba2914117a35895f7c3bfbce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:16.724314 17926 tablet_bootstrap.cc:654] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:16.724560 17924 log.cc:824] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:16.724962 17926 log.cc:824] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:16.725389 17924 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: No bootstrap required, opened a new log
I20260808 07:04:16.725440 17924 ts_tablet_manager.cc:1397] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:16.725855 17926 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: No bootstrap required, opened a new log
I20260808 07:04:16.725915 17926 ts_tablet_manager.cc:1397] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:04:16.726821 17929 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Bootstrap starting.
I20260808 07:04:16.727402 17929 tablet_bootstrap.cc:654] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:16.727941 17929 log.cc:824] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:16.727907 17924 raft_consensus.cc:348] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.728106 17924 raft_consensus.cc:374] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:16.728144 17924 raft_consensus.cc:729] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db72ebb7c719497db36b26ddb70cd021, State: Initialized, Role: FOLLOWER
I20260808 07:04:16.728048 17926 raft_consensus.cc:348] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.728232 17926 raft_consensus.cc:374] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:16.728282 17926 raft_consensus.cc:729] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6a058dd2944a97922c63bc66f3a385, State: Initialized, Role: FOLLOWER
I20260808 07:04:16.728308 17924 consensus_queue.cc:260] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.728394 17926 consensus_queue.cc:260] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.728659 17926 ts_tablet_manager.cc:1428] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:16.728672 17924 ts_tablet_manager.cc:1428] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:16.728705 17929 tablet_bootstrap.cc:492] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: No bootstrap required, opened a new log
I20260808 07:04:16.728705 17491 heartbeater.cc:503] Master 127.24.181.126:44123 was elected leader, sending a full tablet report...
I20260808 07:04:16.728765 17929 ts_tablet_manager.cc:1397] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:16.728811 17897 heartbeater.cc:503] Master 127.24.181.126:44123 was elected leader, sending a full tablet report...
I20260808 07:04:16.730466 17929 raft_consensus.cc:348] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.730624 17929 raft_consensus.cc:374] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:16.730672 17929 raft_consensus.cc:729] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6d7f9a37234173958b4ff5d1670028, State: Initialized, Role: FOLLOWER
I20260808 07:04:16.730785 17929 consensus_queue.cc:260] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.731110 17929 ts_tablet_manager.cc:1428] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:16.731128 17701 heartbeater.cc:503] Master 127.24.181.126:44123 was elected leader, sending a full tablet report...
W20260808 07:04:16.783200 17705 tablet.cc:2369] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:16.863705 17933 raft_consensus.cc:482] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:16.863847 17933 raft_consensus.cc:504] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
W20260808 07:04:16.864184 17492 tablet.cc:2369] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:16.864295 17933 leader_election.cc:288] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc6a058dd2944a97922c63bc66f3a385 (127.24.181.65:34861), db72ebb7c719497db36b26ddb70cd021 (127.24.181.67:42555)
I20260808 07:04:16.867515 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.864464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34861 (local address 127.24.181.66:43245)
0808 07:04:16.864554 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.864567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.864627 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.864848 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.864849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.864851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:16.864935 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.864937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.865648 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.865649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.866622 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.866625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.867231 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.867231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.867328 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.867332 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.867384 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.867425 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 07:04:16.867533 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.864529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:43245 (local address 127.24.181.65:34861)
0808 07:04:16.864606 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.864608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.864620 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:16.864730 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.864732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.864743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.864815 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:16.865010 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.865564 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.866708 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.867150 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.867209 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.867217 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.867408 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.867453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:04:16.867627 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.864464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:42555 (local address 127.24.181.66:34385)
0808 07:04:16.864775 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.864786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:16.864846 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:16.865010 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:16.865010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:16.865012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:16.865079 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.865080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.865758 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.865759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.866506 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:16.866508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.867336 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.867336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:16.867430 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.867434 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:16.867500 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:04:16.867538 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":106,"thread_start_us":125,"threads_started":1}
I20260808 07:04:16.867628 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.864606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:34385 (local address 127.24.181.67:42555)
0808 07:04:16.864658 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.864660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.864858 (+ 198us) server_negotiation.cc:407] Connection header received
0808 07:04:16.864908 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.864910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.864921 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.864977 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:16.865150 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.865632 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.866607 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.867228 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.867305 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.867314 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.867508 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.867551 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:04:16.867892 17426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663f84aba2914117a35895f7c3bfbce6" candidate_uuid: "7d6d7f9a37234173958b4ff5d1670028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6a058dd2944a97922c63bc66f3a385" is_pre_election: true
I20260808 07:04:16.868036 17426 raft_consensus.cc:2454] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6d7f9a37234173958b4ff5d1670028 in term 0.
I20260808 07:04:16.868021 17831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663f84aba2914117a35895f7c3bfbce6" candidate_uuid: "7d6d7f9a37234173958b4ff5d1670028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db72ebb7c719497db36b26ddb70cd021" is_pre_election: true
I20260808 07:04:16.868193 17831 raft_consensus.cc:2454] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6d7f9a37234173958b4ff5d1670028 in term 0.
I20260808 07:04:16.868312 17531 leader_election.cc:302] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6d7f9a37234173958b4ff5d1670028, dc6a058dd2944a97922c63bc66f3a385; no voters:
I20260808 07:04:16.868480 17933 raft_consensus.cc:2781] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:16.868566 17933 raft_consensus.cc:482] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:16.868602 17933 raft_consensus.cc:3037] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:16.869328 17933 raft_consensus.cc:504] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.869477 17933 leader_election.cc:288] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [CANDIDATE]: Term 1 election: Requested vote from peers dc6a058dd2944a97922c63bc66f3a385 (127.24.181.65:34861), db72ebb7c719497db36b26ddb70cd021 (127.24.181.67:42555)
I20260808 07:04:16.869629 17831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663f84aba2914117a35895f7c3bfbce6" candidate_uuid: "7d6d7f9a37234173958b4ff5d1670028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db72ebb7c719497db36b26ddb70cd021"
I20260808 07:04:16.869663 17426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663f84aba2914117a35895f7c3bfbce6" candidate_uuid: "7d6d7f9a37234173958b4ff5d1670028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6a058dd2944a97922c63bc66f3a385"
I20260808 07:04:16.869728 17831 raft_consensus.cc:3037] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:16.869727 17426 raft_consensus.cc:3037] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:16.870471 17831 raft_consensus.cc:2454] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6d7f9a37234173958b4ff5d1670028 in term 1.
I20260808 07:04:16.870602 17426 raft_consensus.cc:2454] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6d7f9a37234173958b4ff5d1670028 in term 1.
I20260808 07:04:16.870657 17532 leader_election.cc:302] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6d7f9a37234173958b4ff5d1670028, db72ebb7c719497db36b26ddb70cd021; no voters:
I20260808 07:04:16.870740 17933 raft_consensus.cc:2781] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:16.870899 17933 raft_consensus.cc:686] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6d7f9a37234173958b4ff5d1670028, State: Running, Role: LEADER
I20260808 07:04:16.871040 17933 consensus_queue.cc:237] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } }
I20260808 07:04:16.871831 17203 catalog_manager.cc:5495] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d6d7f9a37234173958b4ff5d1670028 (127.24.181.66). New cstate: current_term: 1 leader_uuid: "7d6d7f9a37234173958b4ff5d1670028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6d7f9a37234173958b4ff5d1670028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 } health_report { overall_health: UNKNOWN } } }
I20260808 07:04:16.897107 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.894551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.24.181.66:45077)
0808 07:04:16.894616 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:16.894618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:16.894627 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:16.894698 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:16.894699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:16.894748 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:16.894829 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:16.895006 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.895485 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.896278 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:16.896471 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:16.896554 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:16.896737 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:16.896839 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:16.896938 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:16.896957 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:04:16.896977 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:04:16.899047 17831 raft_consensus.cc:1261] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 1 FOLLOWER]: Refusing update from remote peer 7d6d7f9a37234173958b4ff5d1670028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:16.899048 17426 raft_consensus.cc:1261] T 663f84aba2914117a35895f7c3bfbce6 P dc6a058dd2944a97922c63bc66f3a385 [term 1 FOLLOWER]: Refusing update from remote peer 7d6d7f9a37234173958b4ff5d1670028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:16.899389 17933 consensus_queue.cc:1035] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc6a058dd2944a97922c63bc66f3a385" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 34861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:16.899547 17935 consensus_queue.cc:1035] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db72ebb7c719497db36b26ddb70cd021" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 42555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:16.902123 17942 mvcc.cc:205] Tried to move back new op lower bound from 7316163202655404032 to 7316163202544242688. Current Snapshot: MvccSnapshot[applied={T|T < 7316163202655404032}]
I20260808 07:04:16.902174 17938 mvcc.cc:205] Tried to move back new op lower bound from 7316163202655404032 to 7316163202544242688. Current Snapshot: MvccSnapshot[applied={T|T < 7316163202655404032}]
I20260808 07:04:16.902462 17940 mvcc.cc:205] Tried to move back new op lower bound from 7316163202655404032 to 7316163202544242688. Current Snapshot: MvccSnapshot[applied={T|T < 7316163202655404032}]
I20260808 07:04:16.905208 17202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:16.905341 17202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:16.906160 17202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f70787b87e254ec884de8645089ef9b4: Sending DeleteTablet for 3 replicas of tablet 663f84aba2914117a35895f7c3bfbce6
I20260808 07:04:16.906541 17393 tablet_service.cc:1514] Processing DeleteTablet for tablet 663f84aba2914117a35895f7c3bfbce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:16 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:45492
I20260808 07:04:16.906581 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet 663f84aba2914117a35895f7c3bfbce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:16 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:54270
I20260808 07:04:16.906818 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 17294
I20260808 07:04:16.906795 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet 663f84aba2914117a35895f7c3bfbce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:16 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:57230
I20260808 07:04:16.913492 17956 tablet_replica.cc:330] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: stopping tablet replica
I20260808 07:04:16.913625 17956 raft_consensus.cc:2229] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:16.913767 17956 raft_consensus.cc:2258] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:16.914103 17955 tablet_replica.cc:330] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: stopping tablet replica
I20260808 07:04:16.914238 17955 raft_consensus.cc:2229] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:16.914270 17956 ts_tablet_manager.cc:1905] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:16.914356 17955 raft_consensus.cc:2258] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:16.914824 17955 ts_tablet_manager.cc:1905] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:16.916244 17956 ts_tablet_manager.cc:1918] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:16.916381 17956 log.cc:1196] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/663f84aba2914117a35895f7c3bfbce6
I20260808 07:04:16.916749 17956 ts_tablet_manager.cc:1939] T 663f84aba2914117a35895f7c3bfbce6 P 7d6d7f9a37234173958b4ff5d1670028: Deleting consensus metadata
I20260808 07:04:16.917024 17955 ts_tablet_manager.cc:1918] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:16.917186 17955 log.cc:1196] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/663f84aba2914117a35895f7c3bfbce6
I20260808 07:04:16.917253 17183 catalog_manager.cc:4841] TS 7d6d7f9a37234173958b4ff5d1670028 (127.24.181.66:45077): tablet 663f84aba2914117a35895f7c3bfbce6 (table test-table [id=58c0cb962288418388ed611377ba87a6]) successfully deleted
I20260808 07:04:16.917562 17955 ts_tablet_manager.cc:1939] T 663f84aba2914117a35895f7c3bfbce6 P db72ebb7c719497db36b26ddb70cd021: Deleting consensus metadata
I20260808 07:04:16.917989 17184 catalog_manager.cc:4841] TS db72ebb7c719497db36b26ddb70cd021 (127.24.181.67:42555): tablet 663f84aba2914117a35895f7c3bfbce6 (table test-table [id=58c0cb962288418388ed611377ba87a6]) successfully deleted
W20260808 07:04:16.919286 17183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:34861 (error 108)
I20260808 07:04:16.919862 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:16.919711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:34861 (local address 127.0.0.1:45502)
0808 07:04:16.919779 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:16.919826 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:34861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:04:16.919926 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 17501
W20260808 07:04:16.919981 17183 catalog_manager.cc:4584] TS dc6a058dd2944a97922c63bc66f3a385 (127.24.181.65:34861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 663f84aba2914117a35895f7c3bfbce6: Network error: Client connection negotiation failed: client connection to 127.24.181.65:34861: connect: Connection refused (error 111)
I20260808 07:04:16.932699 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 17710
I20260808 07:04:16.943775 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 17157
2026-08-08T07:04:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2303 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T07:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:16Z Disabled control of system clock
I20260808 07:04:16.978900 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46699
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:46699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:17.063822 17963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:17.064038 17963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:17.064059 17963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:17.065136 17963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:17.065172 17963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:17.065186 17963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:17.065197 17963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:17.065210 17963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:17.066473 17963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:46699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:46699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:17.066686 17963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:17.066982 17963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:17.071890 17975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.071903 17972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.071947 17973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.072320 17963 server_base.cc:1034] running on GCE node
I20260808 07:04:17.072537 17963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:17.072774 17963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:17.073909 17963 hybrid_clock.cc:648] HybridClock initialized: now 1786172657073896 us; error 29 us; skew 500 ppm
I20260808 07:04:17.075193 17963 webserver.cc:460] Webserver started at http://127.24.181.126:44027/ using document root <none> and password file <none>
I20260808 07:04:17.075441 17963 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:17.075489 17963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:17.075645 17963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:17.076707 17963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "14279b20f7d2470b9e2058d4301a7aab"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "a2dbdb1d797c89025a751dca8275092f"
server_key_iv: "0e8a174eff9a35bd72e01120b1b410e8"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.077088 17963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "14279b20f7d2470b9e2058d4301a7aab"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "a2dbdb1d797c89025a751dca8275092f"
server_key_iv: "0e8a174eff9a35bd72e01120b1b410e8"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.078452 17963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:17.079288 17982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.079478 17963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:17.079574 17963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "14279b20f7d2470b9e2058d4301a7aab"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "a2dbdb1d797c89025a751dca8275092f"
server_key_iv: "0e8a174eff9a35bd72e01120b1b410e8"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.079651 17963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:17.107237 17963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:17.107582 17963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:17.107757 17963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:17.111925 17963 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:46699
I20260808 07:04:17.111966 18055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:46699 every 8 connection(s)
I20260808 07:04:17.112319 17963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:17.113373 18056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:17.114396 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 17963
I20260808 07:04:17.114492 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:17.114705 25301 external_mini_cluster.cc:1428] Setting key 88f1f1375356a328705f37e0a85f2305
I20260808 07:04:17.116178 18056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: Bootstrap starting.
I20260808 07:04:17.116806 18056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:17.117131 18056 log.cc:824] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:17.117913 18056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: No bootstrap required, opened a new log
I20260808 07:04:17.119818 18056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } }
I20260808 07:04:17.119979 18056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:17.120020 18056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14279b20f7d2470b9e2058d4301a7aab, State: Initialized, Role: FOLLOWER
I20260808 07:04:17.120213 18056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } }
I20260808 07:04:17.120333 18056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:17.120389 18056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:17.120446 18056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:17.120627 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.115318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.24.181.126:46699)
0808 07:04:17.115547 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.115555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.115582 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:04:17.115999 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.116005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.116071 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.116230 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:17.116470 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.117572 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.118447 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.118871 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.119050 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.119776 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:17.119780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:17.119781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:17.119956 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:17.119990 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:17.120001 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:17.120087 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.120240 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.120355 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260808 07:04:17.121119 18056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } }
I20260808 07:04:17.121248 18056 leader_election.cc:302] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14279b20f7d2470b9e2058d4301a7aab; no voters:
I20260808 07:04:17.121435 18056 leader_election.cc:288] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:17.121480 18068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:17.121654 18068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [term 1 LEADER]: Becoming Leader. State: Replica: 14279b20f7d2470b9e2058d4301a7aab, State: Running, Role: LEADER
I20260808 07:04:17.121724 18056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:17.121785 18068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } }
I20260808 07:04:17.122197 18069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14279b20f7d2470b9e2058d4301a7aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } } }
I20260808 07:04:17.122282 18069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:17.122372 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14279b20f7d2470b9e2058d4301a7aab. Latest consensus state: current_term: 1 leader_uuid: "14279b20f7d2470b9e2058d4301a7aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14279b20f7d2470b9e2058d4301a7aab" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 46699 } } }
I20260808 07:04:17.122608 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:17.122822 18080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:17.123382 18080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:17.125105 18080 catalog_manager.cc:1347] Generated new cluster ID: a9943b4649c94134b9e99e8d726f5e5c
I20260808 07:04:17.125161 18080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:17.146863 18080 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:17.147423 18080 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:17.151479 18080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: Generated new TSK 0
I20260808 07:04:17.151667 18080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:17.160817 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--builtin_ntp_servers=127.24.181.84:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:17.243248 18096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:17.243448 18096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:17.243468 18096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:17.244637 18096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:17.244680 18096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:17.244719 18096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:17.246073 18096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:17.246310 18096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:17.246623 18096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:17.252496 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.252539 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.252667 18096 server_base.cc:1034] running on GCE node
W20260808 07:04:17.252498 18106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.252985 18096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:17.253289 18096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:17.254467 18096 hybrid_clock.cc:648] HybridClock initialized: now 1786172657254441 us; error 37 us; skew 500 ppm
I20260808 07:04:17.255838 18096 webserver.cc:460] Webserver started at http://127.24.181.65:43873/ using document root <none> and password file <none>
I20260808 07:04:17.256108 18096 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:17.256156 18096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:17.256322 18096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:17.257407 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "d72ca76e6ff74d6288ae3f17e7d53a21"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "9288d2705f3a3e85841715ab9802da3e"
server_key_iv: "90f646403d5e256a1f1e7e716ac89568"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.257778 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "d72ca76e6ff74d6288ae3f17e7d53a21"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "9288d2705f3a3e85841715ab9802da3e"
server_key_iv: "90f646403d5e256a1f1e7e716ac89568"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.259173 18096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:17.259903 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.260071 18096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:17.260174 18096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "d72ca76e6ff74d6288ae3f17e7d53a21"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "9288d2705f3a3e85841715ab9802da3e"
server_key_iv: "90f646403d5e256a1f1e7e716ac89568"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.260264 18096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:17.277532 18096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:17.277875 18096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:17.278038 18096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:17.278285 18096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:17.278615 18096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:17.278673 18096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.278714 18096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:17.278743 18096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.285254 18096 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:44531
I20260808 07:04:17.285283 18287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:44531 every 8 connection(s)
I20260808 07:04:17.285616 18096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:17.286581 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 18096
I20260808 07:04:17.286688 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:17.286886 25301 external_mini_cluster.cc:1428] Setting key b8a2f85a751014afae3d3f81b228f014
I20260808 07:04:17.287954 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--builtin_ntp_servers=127.24.181.84:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:04:17.291759 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.286365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:41553 (local address 127.24.181.126:46699)
0808 07:04:17.286433 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.286436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.286732 (+ 296us) server_negotiation.cc:407] Connection header received
0808 07:04:17.287435 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.287436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.287446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.287502 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:17.288382 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.289141 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.290092 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.290424 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.290507 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.291398 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:17.291400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:17.291400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:17.291406 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:17.291424 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:17.291428 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:17.291489 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.291613 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.291639 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:04:17.291924 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.286355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46699 (local address 127.24.181.65:41553)
0808 07:04:17.286555 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.286597 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.287324 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.287589 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.287593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.287603 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:17.288281 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.288287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.289240 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.289241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.289999 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.290001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.290524 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.290525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.290633 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.291244 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:17.291245 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:17.291274 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:17.291298 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:17.291503 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:17.291509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:17.291511 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.291589 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.291668 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260808 07:04:17.292462 18288 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46699
I20260808 07:04:17.292572 18288 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:17.292761 18288 heartbeater.cc:511] Master 127.24.181.126:46699 requested a full tablet report, sending...
I20260808 07:04:17.293357 18007 ts_manager.cc:194] Registered new tserver with Master: d72ca76e6ff74d6288ae3f17e7d53a21 (127.24.181.65:44531)
I20260808 07:04:17.294399 18007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.65:41553
W20260808 07:04:17.371843 18291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:17.372043 18291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:17.372066 18291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:17.373124 18291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:17.373157 18291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:17.373181 18291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:17.374552 18291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:17.374742 18291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:17.375012 18291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:17.380009 18301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.380143 18291 server_base.cc:1034] running on GCE node
W20260808 07:04:17.379983 18304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.379994 18298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.380425 18291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:17.380616 18291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:17.381819 18291 hybrid_clock.cc:648] HybridClock initialized: now 1786172657381802 us; error 29 us; skew 500 ppm
I20260808 07:04:17.383145 18291 webserver.cc:460] Webserver started at http://127.24.181.66:36705/ using document root <none> and password file <none>
I20260808 07:04:17.383397 18291 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:17.383443 18291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:17.383591 18291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:17.384656 18291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "5a92b756be2fe92a5d87d836febaa60e"
server_key_iv: "1ad8d67c1f7159e53477bd7d5aa76259"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.385063 18291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "5a92b756be2fe92a5d87d836febaa60e"
server_key_iv: "1ad8d67c1f7159e53477bd7d5aa76259"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.386380 18291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:17.387187 18312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.387370 18291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:17.387472 18291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "5a92b756be2fe92a5d87d836febaa60e"
server_key_iv: "1ad8d67c1f7159e53477bd7d5aa76259"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.387555 18291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:17.412488 18291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:17.412858 18291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:17.413030 18291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:17.413301 18291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:17.413650 18291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:17.413719 18291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.413760 18291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:17.413789 18291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.419646 18291 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:42985
I20260808 07:04:17.419691 18480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:42985 every 8 connection(s)
I20260808 07:04:17.420027 18291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:17.424418 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 18291
I20260808 07:04:17.424554 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:17.424775 25301 external_mini_cluster.cc:1428] Setting key 70b89d7c9405c30077adf21cd4908c24
I20260808 07:04:17.425894 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--builtin_ntp_servers=127.24.181.84:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:04:17.426343 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.420740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:51083 (local address 127.24.181.126:46699)
0808 07:04:17.420829 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.420831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.421097 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:04:17.421580 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.421580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.421591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.421703 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:17.422452 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.423301 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.424600 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.424850 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.424929 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.426003 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:17.426005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:17.426006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:17.426011 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:17.426034 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:17.426039 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:17.426103 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.426236 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.426262 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:04:17.426591 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.420707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46699 (local address 127.24.181.66:51083)
0808 07:04:17.420929 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.420992 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.421491 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.421799 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.421803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.421811 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:17.422372 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.422377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.423412 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.423413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.424512 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.424515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.424955 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.424956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.425138 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.425818 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:17.425818 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:17.425858 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:17.425888 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:17.426132 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:17.426140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:17.426143 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.426237 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.426356 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
I20260808 07:04:17.427105 18481 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46699
I20260808 07:04:17.427198 18481 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:17.427409 18481 heartbeater.cc:511] Master 127.24.181.126:46699 requested a full tablet report, sending...
I20260808 07:04:17.427894 18007 ts_manager.cc:194] Registered new tserver with Master: 8bd71a83f7b14220a3ca9d6a0a2967c3 (127.24.181.66:42985)
I20260808 07:04:17.428290 18007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.66:51083
W20260808 07:04:17.510094 18484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:17.510310 18484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:17.510334 18484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:17.511620 18484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:17.511664 18484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:17.511691 18484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:17.513130 18484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:17.513412 18484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:17.513749 18484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:17.519125 18497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.519137 18500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:17.519137 18494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:17.519572 18484 server_base.cc:1034] running on GCE node
I20260808 07:04:17.519845 18484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:17.520105 18484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:17.521281 18484 hybrid_clock.cc:648] HybridClock initialized: now 1786172657521278 us; error 59 us; skew 500 ppm
I20260808 07:04:17.522778 18484 webserver.cc:460] Webserver started at http://127.24.181.67:44447/ using document root <none> and password file <none>
I20260808 07:04:17.523102 18484 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:17.523182 18484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:17.523336 18484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:17.524530 18484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "922134f069064244b5eef40cb2751d70"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "c3d5605d7154f22e60596216e0665e5d"
server_key_iv: "2b949cdb36097776468cb9d4498c8be6"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.524917 18484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "922134f069064244b5eef40cb2751d70"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "c3d5605d7154f22e60596216e0665e5d"
server_key_iv: "2b949cdb36097776468cb9d4498c8be6"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.526309 18484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:17.527160 18506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.527359 18484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:17.527465 18484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "922134f069064244b5eef40cb2751d70"
format_stamp: "Formatted at 2026-08-08 07:04:17 on dist-test-slave-nfb5"
server_key: "c3d5605d7154f22e60596216e0665e5d"
server_key_iv: "2b949cdb36097776468cb9d4498c8be6"
server_key_version: "encryptionkey@0"
I20260808 07:04:17.527549 18484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:17.535184 18484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:17.535467 18484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:17.535620 18484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:17.535844 18484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:17.536216 18484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:17.536278 18484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.536319 18484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:17.536348 18484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:17.542446 18484 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:39527
I20260808 07:04:17.542492 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:39527 every 8 connection(s)
I20260808 07:04:17.542814 18484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:17.548100 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.543580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:53265 (local address 127.24.181.126:46699)
0808 07:04:17.543660 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.543662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.543907 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:04:17.544443 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.544444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.544457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.544510 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:17.545232 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.545718 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.546666 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.546879 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.547006 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.547807 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:17.547809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:17.547810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:17.547815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:17.547830 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:17.547834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:17.547890 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.547996 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.548022 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:04:17.548334 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.543550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:46699 (local address 127.24.181.67:53265)
0808 07:04:17.543785 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.543817 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.544327 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.544580 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.544583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.544592 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:17.545115 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.545119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.545829 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.545830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.546566 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.546568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.547020 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.547021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.547145 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.547616 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:17.547616 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:17.547642 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:17.547681 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:17.547893 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:17.547899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:17.547903 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.547984 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.548085 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
I20260808 07:04:17.548849 18669 heartbeater.cc:347] Connected to a master server at 127.24.181.126:46699
I20260808 07:04:17.548929 18669 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:17.549127 18669 heartbeater.cc:511] Master 127.24.181.126:46699 requested a full tablet report, sending...
I20260808 07:04:17.549628 18007 ts_manager.cc:194] Registered new tserver with Master: 922134f069064244b5eef40cb2751d70 (127.24.181.67:39527)
I20260808 07:04:17.550040 18007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.67:53265
I20260808 07:04:17.551987 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 18484
I20260808 07:04:17.552105 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:17.552306 25301 external_mini_cluster.cc:1428] Setting key e9ff4a775b7ed8044a73483cca4c7477
I20260808 07:04:17.553233 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:17.557331 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.554622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.24.181.126:46699)
0808 07:04:17.554678 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.554680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.554818 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:04:17.554859 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.554860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.554868 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.555011 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:17.555183 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.555626 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.556318 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.556481 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.556529 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.557139 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:17.557140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:17.557140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:17.557144 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:17.557157 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:17.557160 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:17.557206 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.557265 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.557283 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:04:17.559018 18007 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:17.560570 18007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:17.567464 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.563367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.24.181.67:39527)
0808 07:04:17.563605 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.563609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.563618 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:17.563654 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.563656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.563669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.563933 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:17.563986 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.565074 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.566299 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.567031 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.567142 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.567246 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.567331 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.567380 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260808 07:04:17.567458 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.562897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:39527 (local address 127.0.0.1:41458)
0808 07:04:17.563289 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.563299 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.563352 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.563815 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.563815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.563817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:17.563860 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.563862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.565172 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.565173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.566214 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.566216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.567143 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.567144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.567222 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.567239 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.567320 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.567373 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":4,"thread_start_us":102,"threads_started":1}
I20260808 07:04:17.567672 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.563125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42985 (local address 127.0.0.1:33516)
0808 07:04:17.563367 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.563378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.563468 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.563837 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.563838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.563840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:17.563883 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.563885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.564888 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.564889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.566137 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.566140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.567238 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.567239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.567310 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.567316 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.567369 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.567409 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":81,"threads_started":1}
I20260808 07:04:17.568107 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.563284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.24.181.66:42985)
0808 07:04:17.563594 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.563598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.563609 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:17.563648 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.563649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.563665 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.563744 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:17.563983 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.564796 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.566224 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.567046 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.567109 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.567202 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.567501 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.568023 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":88,"threads_started":1}
I20260808 07:04:17.568228 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.562897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44531 (local address 127.0.0.1:36854)
0808 07:04:17.563041 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.563084 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.563157 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.563748 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.563751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.563754 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:17.563819 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.563821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.566314 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.566315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.567025 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.567026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.568004 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.568005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.568068 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.568072 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.568105 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.568133 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260808 07:04:17.568151 18580 tablet_service.cc:1467] Processing CreateTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a (DEFAULT_TABLE table=test-table [id=4c2c8ed47b4047b09b52fb69152836fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:17.568532 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668a04b1d05d4e1bb6da15d35511cc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:17.568668 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.563045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.24.181.65:44531)
0808 07:04:17.563536 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.563542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.563553 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:17.563602 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.563605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.563622 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.563906 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:17.563970 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.565074 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.567111 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.567909 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.568398 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.568497 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.568521 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.568560 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":104,"threads_started":1}
I20260808 07:04:17.568729 18380 tablet_service.cc:1467] Processing CreateTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a (DEFAULT_TABLE table=test-table [id=4c2c8ed47b4047b09b52fb69152836fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:17.569075 18380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668a04b1d05d4e1bb6da15d35511cc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:17.569253 18186 tablet_service.cc:1467] Processing CreateTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a (DEFAULT_TABLE table=test-table [id=4c2c8ed47b4047b09b52fb69152836fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:17.569552 18186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668a04b1d05d4e1bb6da15d35511cc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:17.572129 18691 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Bootstrap starting.
I20260808 07:04:17.572570 18694 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Bootstrap starting.
I20260808 07:04:17.572577 18692 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Bootstrap starting.
I20260808 07:04:17.572902 18691 tablet_bootstrap.cc:654] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:17.573240 18691 log.cc:824] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:17.573279 18694 tablet_bootstrap.cc:654] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:17.573279 18692 tablet_bootstrap.cc:654] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:17.573612 18692 log.cc:824] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:17.573613 18694 log.cc:824] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:17.574076 18691 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: No bootstrap required, opened a new log
I20260808 07:04:17.574162 18691 ts_tablet_manager.cc:1397] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:17.574452 18694 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: No bootstrap required, opened a new log
I20260808 07:04:17.574529 18694 ts_tablet_manager.cc:1397] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:17.574733 18692 tablet_bootstrap.cc:492] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: No bootstrap required, opened a new log
I20260808 07:04:17.574801 18692 ts_tablet_manager.cc:1397] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:17.576363 18691 raft_consensus.cc:348] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.576545 18691 raft_consensus.cc:374] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:17.576591 18691 raft_consensus.cc:729] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922134f069064244b5eef40cb2751d70, State: Initialized, Role: FOLLOWER
I20260808 07:04:17.576704 18691 consensus_queue.cc:260] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.576975 18691 ts_tablet_manager.cc:1428] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:17.577121 18669 heartbeater.cc:503] Master 127.24.181.126:46699 was elected leader, sending a full tablet report...
I20260808 07:04:17.577303 18694 raft_consensus.cc:348] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.577358 18692 raft_consensus.cc:348] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.577466 18694 raft_consensus.cc:374] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:17.577476 18692 raft_consensus.cc:374] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:17.577513 18694 raft_consensus.cc:729] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bd71a83f7b14220a3ca9d6a0a2967c3, State: Initialized, Role: FOLLOWER
I20260808 07:04:17.577513 18692 raft_consensus.cc:729] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72ca76e6ff74d6288ae3f17e7d53a21, State: Initialized, Role: FOLLOWER
I20260808 07:04:17.577620 18692 consensus_queue.cc:260] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.577620 18694 consensus_queue.cc:260] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.577893 18692 ts_tablet_manager.cc:1428] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:17.577893 18694 ts_tablet_manager.cc:1428] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:04:17.578009 18288 heartbeater.cc:503] Master 127.24.181.126:46699 was elected leader, sending a full tablet report...
I20260808 07:04:17.578009 18481 heartbeater.cc:503] Master 127.24.181.126:46699 was elected leader, sending a full tablet report...
W20260808 07:04:17.670732 18482 tablet.cc:2369] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:17.786337 18289 tablet.cc:2369] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:17.793375 18671 tablet.cc:2369] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:17.861807 18699 raft_consensus.cc:482] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:17.861943 18699 raft_consensus.cc:504] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.862313 18699 leader_election.cc:288] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72ca76e6ff74d6288ae3f17e7d53a21 (127.24.181.65:44531), 8bd71a83f7b14220a3ca9d6a0a2967c3 (127.24.181.66:42985)
I20260808 07:04:17.865442 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.862587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:59823 (local address 127.24.181.65:44531)
0808 07:04:17.862637 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.862639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.862656 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:04:17.862683 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.862684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.862693 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.862780 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:17.863034 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.863600 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.864727 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.865075 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.865171 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.865181 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.865330 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.865369 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:04:17.865442 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.862486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44531 (local address 127.24.181.67:59823)
0808 07:04:17.862550 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.862564 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.862613 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.862801 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.862802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.862804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:17.862935 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.862938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.863787 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.863789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.864622 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.864625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.865171 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.865172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.865253 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.865256 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.865324 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.865369 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:04:17.865535 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.862795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:57883 (local address 127.24.181.66:42985)
0808 07:04:17.862863 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.862865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.862936 (+ 71us) server_negotiation.cc:407] Connection header received
0808 07:04:17.862991 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.862992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.863001 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.863059 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:17.863234 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.863783 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.864792 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.865153 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.865242 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.865250 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.865420 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.865459 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:04:17.865558 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.862632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42985 (local address 127.24.181.67:57883)
0808 07:04:17.862846 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.862855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:17.862930 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:17.863085 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:17.863085 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:17.863087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:17.863158 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.863160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.863875 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.863876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.864679 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:17.864682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.865243 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.865244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:17.865322 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.865326 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:17.865412 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:04:17.865450 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":138,"threads_started":1}
I20260808 07:04:17.865818 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668a04b1d05d4e1bb6da15d35511cc1a" candidate_uuid: "922134f069064244b5eef40cb2751d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" is_pre_election: true
I20260808 07:04:17.865967 18413 raft_consensus.cc:2454] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922134f069064244b5eef40cb2751d70 in term 0.
I20260808 07:04:17.865895 18215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668a04b1d05d4e1bb6da15d35511cc1a" candidate_uuid: "922134f069064244b5eef40cb2751d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" is_pre_election: true
I20260808 07:04:17.866086 18215 raft_consensus.cc:2454] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922134f069064244b5eef40cb2751d70 in term 0.
I20260808 07:04:17.866223 18507 leader_election.cc:302] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bd71a83f7b14220a3ca9d6a0a2967c3, 922134f069064244b5eef40cb2751d70; no voters:
I20260808 07:04:17.866376 18699 raft_consensus.cc:2781] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:17.866456 18699 raft_consensus.cc:482] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:17.866473 18699 raft_consensus.cc:3037] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:17.867277 18699 raft_consensus.cc:504] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.867416 18699 leader_election.cc:288] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [CANDIDATE]: Term 1 election: Requested vote from peers d72ca76e6ff74d6288ae3f17e7d53a21 (127.24.181.65:44531), 8bd71a83f7b14220a3ca9d6a0a2967c3 (127.24.181.66:42985)
I20260808 07:04:17.867607 18215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668a04b1d05d4e1bb6da15d35511cc1a" candidate_uuid: "922134f069064244b5eef40cb2751d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21"
I20260808 07:04:17.867662 18413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668a04b1d05d4e1bb6da15d35511cc1a" candidate_uuid: "922134f069064244b5eef40cb2751d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3"
I20260808 07:04:17.867691 18215 raft_consensus.cc:3037] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:17.867764 18413 raft_consensus.cc:3037] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:17.868430 18215 raft_consensus.cc:2454] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922134f069064244b5eef40cb2751d70 in term 1.
I20260808 07:04:17.868630 18507 leader_election.cc:302] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 922134f069064244b5eef40cb2751d70, d72ca76e6ff74d6288ae3f17e7d53a21; no voters:
I20260808 07:04:17.868686 18413 raft_consensus.cc:2454] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922134f069064244b5eef40cb2751d70 in term 1.
I20260808 07:04:17.868741 18699 raft_consensus.cc:2781] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:17.868917 18699 raft_consensus.cc:686] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 1 LEADER]: Becoming Leader. State: Replica: 922134f069064244b5eef40cb2751d70, State: Running, Role: LEADER
I20260808 07:04:17.869021 18699 consensus_queue.cc:237] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } }
I20260808 07:04:17.869789 18007 catalog_manager.cc:5495] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 922134f069064244b5eef40cb2751d70 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "922134f069064244b5eef40cb2751d70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "922134f069064244b5eef40cb2751d70" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 39527 } health_report { overall_health: HEALTHY } } }
I20260808 07:04:17.923697 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.920955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.24.181.67:39527)
0808 07:04:17.921042 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:17.921045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:17.921055 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:17.921149 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:17.921150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:17.921172 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:17.921265 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:17.921403 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.921999 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.922791 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:17.923084 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:17.923149 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:17.923310 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:17.923423 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:17.923524 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:17.923544 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 07:04:17.923561 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:04:17.925568 18413 raft_consensus.cc:1261] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 1 FOLLOWER]: Refusing update from remote peer 922134f069064244b5eef40cb2751d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:17.925626 18215 raft_consensus.cc:1261] T 668a04b1d05d4e1bb6da15d35511cc1a P d72ca76e6ff74d6288ae3f17e7d53a21 [term 1 FOLLOWER]: Refusing update from remote peer 922134f069064244b5eef40cb2751d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:17.925961 18699 consensus_queue.cc:1035] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bd71a83f7b14220a3ca9d6a0a2967c3" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:17.926123 18706 consensus_queue.cc:1035] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72ca76e6ff74d6288ae3f17e7d53a21" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 44531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:17.928987 18709 mvcc.cc:205] Tried to move back new op lower bound from 7316163206860423168 to 7316163206632034304. Current Snapshot: MvccSnapshot[applied={T|T < 7316163206860423168 or (T in {7316163206860423168})}]
I20260808 07:04:17.931094 18006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:17.931197 18006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:17.932036 18006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14279b20f7d2470b9e2058d4301a7aab: Sending DeleteTablet for 3 replicas of tablet 668a04b1d05d4e1bb6da15d35511cc1a
I20260808 07:04:17.932384 18580 tablet_service.cc:1514] Processing DeleteTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:17 UTC) from {username='slave'} at 127.0.0.1:41458
I20260808 07:04:17.932421 18186 tablet_service.cc:1514] Processing DeleteTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:17 UTC) from {username='slave'} at 127.0.0.1:36854
I20260808 07:04:17.932597 18720 tablet_replica.cc:330] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: stopping tablet replica
I20260808 07:04:17.932740 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 18096
I20260808 07:04:17.932775 18720 raft_consensus.cc:2229] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:17.932878 18720 raft_consensus.cc:2258] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:17.939352 18380 tablet_service.cc:1514] Processing DeleteTablet for tablet 668a04b1d05d4e1bb6da15d35511cc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:17 UTC) from {username='slave'} at 127.0.0.1:33516
I20260808 07:04:17.939656 18722 tablet_replica.cc:330] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: stopping tablet replica
I20260808 07:04:17.939770 18722 raft_consensus.cc:2229] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:17.939870 18722 raft_consensus.cc:2258] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:17.941380 18722 ts_tablet_manager.cc:1905] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:17.941573 18720 ts_tablet_manager.cc:1905] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:17.943408 18720 ts_tablet_manager.cc:1918] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:17.943408 18722 ts_tablet_manager.cc:1918] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:17.943488 18722 log.cc:1196] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/668a04b1d05d4e1bb6da15d35511cc1a
I20260808 07:04:17.943532 18720 log.cc:1196] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/668a04b1d05d4e1bb6da15d35511cc1a
I20260808 07:04:17.943804 18722 ts_tablet_manager.cc:1939] T 668a04b1d05d4e1bb6da15d35511cc1a P 8bd71a83f7b14220a3ca9d6a0a2967c3: Deleting consensus metadata
I20260808 07:04:17.943846 18720 ts_tablet_manager.cc:1939] T 668a04b1d05d4e1bb6da15d35511cc1a P 922134f069064244b5eef40cb2751d70: Deleting consensus metadata
I20260808 07:04:17.944249 17986 catalog_manager.cc:4841] TS 922134f069064244b5eef40cb2751d70 (127.24.181.67:39527): tablet 668a04b1d05d4e1bb6da15d35511cc1a (table test-table [id=4c2c8ed47b4047b09b52fb69152836fd]) successfully deleted
I20260808 07:04:17.944474 17984 catalog_manager.cc:4841] TS 8bd71a83f7b14220a3ca9d6a0a2967c3 (127.24.181.66:42985): tablet 668a04b1d05d4e1bb6da15d35511cc1a (table test-table [id=4c2c8ed47b4047b09b52fb69152836fd]) successfully deleted
W20260808 07:04:17.946622 17984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:44531 (error 108)
I20260808 07:04:17.946882 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 18291
I20260808 07:04:17.948314 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:17.946995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:44531 (local address 127.0.0.1:36860)
0808 07:04:17.948229 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:17.948272 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1212}
W20260808 07:04:17.964707 17984 catalog_manager.cc:4584] TS d72ca76e6ff74d6288ae3f17e7d53a21 (127.24.181.65:44531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 668a04b1d05d4e1bb6da15d35511cc1a: Network error: Client connection negotiation failed: client connection to 127.24.181.65:44531: connect: Connection refused (error 111)
I20260808 07:04:17.969125 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 18484
I20260808 07:04:17.978405 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 17963
2026-08-08T07:04:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1031 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:18 dist-test-slave-nfb5 krb5kdc[18729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:18 dist-test-slave-nfb5 krb5kdc[18729](info): set up 2 sockets
Aug 08 07:04:18 dist-test-slave-nfb5 krb5kdc[18729](info): commencing operation
krb5kdc: starting...
W20260808 07:04:19.198164 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:19.325824 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:42501
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:42501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:19.408923 18752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:19.409135 18752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:19.409161 18752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:19.410704 18752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:19.410755 18752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:19.410771 18752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:19.410784 18752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:19.410960 18752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:19.412667 18752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:42501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:42501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:19.412943 18752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:19.413199 18752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:19.418329 18761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.418357 18758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.418362 18764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:19.418414 18752 server_base.cc:1034] running on GCE node
I20260808 07:04:19.418802 18752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:19.419133 18752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:19.420324 18752 hybrid_clock.cc:648] HybridClock initialized: now 1786172659420320 us; error 39 us; skew 500 ppm
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:19.422848 18752 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:19.423333 18752 webserver.cc:460] Webserver started at http://127.24.181.126:41587/ using document root <none> and password file <none>
I20260808 07:04:19.423592 18752 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:19.423666 18752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:19.423820 18752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:19.424903 18752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "4ba8023aa4354f03804905aaaa9a647f"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "a8545edf5726e9c0d6b2393c07a9c3ed"
server_key_iv: "d1ea3c84faa582bbd9a738f46592c72d"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.425271 18752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "4ba8023aa4354f03804905aaaa9a647f"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "a8545edf5726e9c0d6b2393c07a9c3ed"
server_key_iv: "d1ea3c84faa582bbd9a738f46592c72d"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.426715 18752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:19.427606 18774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.427776 18752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:19.427886 18752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "4ba8023aa4354f03804905aaaa9a647f"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "a8545edf5726e9c0d6b2393c07a9c3ed"
server_key_iv: "d1ea3c84faa582bbd9a738f46592c72d"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.427971 18752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:19.439100 18752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:19.440968 18752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:19.441145 18752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:19.445215 18752 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:42501
I20260808 07:04:19.445267 18849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:42501 every 8 connection(s)
I20260808 07:04:19.445662 18752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:19.446573 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:19.448886 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: Bootstrap starting.
I20260808 07:04:19.449438 18850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:19.449680 18850 log.cc:824] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:19.450363 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: No bootstrap required, opened a new log
I20260808 07:04:19.451108 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 18752
I20260808 07:04:19.451237 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:19.451447 25301 external_mini_cluster.cc:1428] Setting key 827e74f57d0cc3eafc9813162d83e9c7
I20260808 07:04:19.452589 18850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } }
I20260808 07:04:19.452770 18850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:19.452811 18850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba8023aa4354f03804905aaaa9a647f, State: Initialized, Role: FOLLOWER
I20260808 07:04:19.452958 18850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } }
I20260808 07:04:19.453043 18850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:19.453080 18850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:19.453130 18850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:19.454113 18850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } }
I20260808 07:04:19.454239 18850 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba8023aa4354f03804905aaaa9a647f; no voters:
I20260808 07:04:19.454438 18850 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:19.454457 18855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:19.454571 18855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [term 1 LEADER]: Becoming Leader. State: Replica: 4ba8023aa4354f03804905aaaa9a647f, State: Running, Role: LEADER
I20260808 07:04:19.454707 18850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:19.454722 18855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } }
I20260808 07:04:19.455199 18857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba8023aa4354f03804905aaaa9a647f. Latest consensus state: current_term: 1 leader_uuid: "4ba8023aa4354f03804905aaaa9a647f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } } }
I20260808 07:04:19.455289 18857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:19.455531 18856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba8023aa4354f03804905aaaa9a647f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba8023aa4354f03804905aaaa9a647f" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 42501 } } }
I20260808 07:04:19.455893 18856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:19.456099 18864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:19.456637 18864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:19.458343 18864 catalog_manager.cc:1347] Generated new cluster ID: 263f618e267c42d494f093817e4b44ea
I20260808 07:04:19.458395 18864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:19.463400 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:19.451989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.24.181.126:42501)
0808 07:04:19.452164 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:19.452172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:19.452196 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:04:19.452571 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:19.452577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:19.452621 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:19.452728 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:19.453615 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.454640 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.455690 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.456388 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.456806 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:19.459354 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:19.459358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:19.459358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:19.459370 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:19.461376 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.461797 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.461800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.461801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.461926 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.462164 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.462165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.462165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.462448 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:19.463008 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:19.463074 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:04:19.463203 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260808 07:04:19.474573 18864 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:19.475183 18864 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:19.485273 18864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: Generated new TSK 0
I20260808 07:04:19.485497 18864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:04:19.517386 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:19.620121 18886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:19.620347 18886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:19.620374 18886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:19.621569 18886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:19.621613 18886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:19.621640 18886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:19.623248 18886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:19.623507 18886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:19.623775 18886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:19.629062 18896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.629062 18893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.629160 18894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:19.629381 18886 server_base.cc:1034] running on GCE node
I20260808 07:04:19.629612 18886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:19.629844 18886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:19.630998 18886 hybrid_clock.cc:648] HybridClock initialized: now 1786172659630977 us; error 28 us; skew 500 ppm
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:19.633798 18886 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:04:19.634259 18886 webserver.cc:460] Webserver started at http://127.24.181.65:46681/ using document root <none> and password file <none>
I20260808 07:04:19.634516 18886 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:19.634591 18886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:19.634785 18886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:19.635924 18886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "49829d49c7ed46dd819e07b55fe808fc"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "872a05fabb6504c846c770cc6356ceae"
server_key_iv: "8aaea8a6714dddec1f9f1ce88d043a85"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.636301 18886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "49829d49c7ed46dd819e07b55fe808fc"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "872a05fabb6504c846c770cc6356ceae"
server_key_iv: "8aaea8a6714dddec1f9f1ce88d043a85"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.637625 18886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:19.638392 18905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.638628 18886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:19.638756 18886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "49829d49c7ed46dd819e07b55fe808fc"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "872a05fabb6504c846c770cc6356ceae"
server_key_iv: "8aaea8a6714dddec1f9f1ce88d043a85"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.638857 18886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:19.653811 18886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:19.656160 18886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:19.656329 18886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:19.656612 18886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:19.656982 18886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:19.657042 18886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.657085 18886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:19.657114 18886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.663251 18886 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:41593
I20260808 07:04:19.663321 19062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:41593 every 8 connection(s)
I20260808 07:04:19.663641 18886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:19.663854 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 18886
I20260808 07:04:19.664000 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:19.664238 25301 external_mini_cluster.cc:1428] Setting key ad002fd0914f2ee26ced5ae6497ce484
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
I20260808 07:04:19.675231 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:19.664530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:48027 (local address 127.24.181.126:42501)
0808 07:04:19.664632 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:19.664635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:19.664860 (+ 225us) server_negotiation.cc:407] Connection header received
0808 07:04:19.665568 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:19.665569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:19.665578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:19.665628 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:19.666981 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.667784 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.669298 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.669507 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.669565 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:19.672085 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:19.672087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:19.672088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:19.672096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:19.673805 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.674213 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.674214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.674215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.674280 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.674537 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.674538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.674538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.674695 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:19.674775 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:19.675078 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 07:04:19.675155 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:04:19.675405 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:19.664398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42501 (local address 127.24.181.65:48027)
0808 07:04:19.664723 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:19.664770 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:19.665461 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:19.665703 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:19.665707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:19.666569 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:19.666855 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:19.666862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.667902 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.667903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:19.669174 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:19.669177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.669605 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.669606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:19.669769 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:19.670296 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:19.670307 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:19.671981 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:19.673894 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:19.673901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:19.673902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:19.674120 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:19.674355 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:19.674355 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:19.674356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:19.674460 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:19.674768 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:19.674775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:19.674902 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 07:04:19.675053 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 07:04:19.675179 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260808 07:04:19.675997 19063 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42501
I20260808 07:04:19.676095 19063 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:19.676326 19063 heartbeater.cc:511] Master 127.24.181.126:42501 requested a full tablet report, sending...
I20260808 07:04:19.676949 18796 ts_manager.cc:194] Registered new tserver with Master: 49829d49c7ed46dd819e07b55fe808fc (127.24.181.65:41593)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
I20260808 07:04:19.677773 18796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:48027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:04:19.693508 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:19.782668 19071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:19.782924 19071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:19.783002 19071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:19.784283 19071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:19.784365 19071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:19.784420 19071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:19.786048 19071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:19.786342 19071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:19.786595 19071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:19.792085 19080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.792098 19077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.792098 19076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:19.792408 19071 server_base.cc:1034] running on GCE node
I20260808 07:04:19.792606 19071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:19.792882 19071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:19.794075 19071 hybrid_clock.cc:648] HybridClock initialized: now 1786172659794065 us; error 39 us; skew 500 ppm
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:19.800777 19071 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:04:19.801313 19071 webserver.cc:460] Webserver started at http://127.24.181.66:39305/ using document root <none> and password file <none>
I20260808 07:04:19.801573 19071 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:19.801646 19071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:19.801808 19071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:19.802898 19071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "4df0f303e8474f8a9bc702a276ef6f5a"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "684360af0e95675bbf1210132c1a8181"
server_key_iv: "0fe6bb88852e985f7ed69aeb2cc790ce"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.803351 19071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "4df0f303e8474f8a9bc702a276ef6f5a"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "684360af0e95675bbf1210132c1a8181"
server_key_iv: "0fe6bb88852e985f7ed69aeb2cc790ce"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.804718 19071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:19.805471 19091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.805656 19071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:19.805765 19071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "4df0f303e8474f8a9bc702a276ef6f5a"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "684360af0e95675bbf1210132c1a8181"
server_key_iv: "0fe6bb88852e985f7ed69aeb2cc790ce"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.805850 19071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:19.834231 19071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:19.836251 19071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:19.836431 19071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:19.836681 19071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:19.837023 19071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:19.837085 19071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.837127 19071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:19.837148 19071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.843461 19071 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:42207
I20260808 07:04:19.843518 19256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:42207 every 8 connection(s)
I20260808 07:04:19.843825 19071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:19.849192 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 19071
I20260808 07:04:19.849345 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:19.849575 25301 external_mini_cluster.cc:1428] Setting key 42694a8524bf4d7195383a390630abab
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:19.853230 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:19.844569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:35569 (local address 127.24.181.126:42501)
0808 07:04:19.844660 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:19.844663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:19.844831 (+ 168us) server_negotiation.cc:407] Connection header received
0808 07:04:19.845320 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:19.845321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:19.845332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:19.845381 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:19.846288 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.846781 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.847724 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.847908 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.847983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:19.850377 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:19.850378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:19.850379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:19.850385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:19.851800 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.852230 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.852231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.852231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.852294 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:19.852558 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:19.852559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:19.852559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:19.852675 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:19.852755 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:19.853039 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:04:19.853128 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 07:04:19.853410 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:19.844511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42501 (local address 127.24.181.66:35569)
0808 07:04:19.844709 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:19.844746 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:19.845240 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:19.845442 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:19.845445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:19.846015 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:19.846208 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:19.846213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.846870 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.846870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:19.847646 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:19.847648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:19.848008 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:19.848009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:19.848113 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:19.848668 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:19.848677 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:19.850292 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:19.851900 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:19.851907 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:19.851909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:19.852136 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:19.852366 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:19.852367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:19.852367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:19.852485 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:19.852758 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:19.852766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:19.852905 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:04:19.853016 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:04:19.853177 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
I20260808 07:04:19.853863 19257 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42501
I20260808 07:04:19.853953 19257 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:19.854146 19257 heartbeater.cc:511] Master 127.24.181.126:42501 requested a full tablet report, sending...
I20260808 07:04:19.854573 18796 ts_manager.cc:194] Registered new tserver with Master: 4df0f303e8474f8a9bc702a276ef6f5a (127.24.181.66:42207)
I20260808 07:04:19.855162 18796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:35569
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:04:19.878182 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:39673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:19.970028 19263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:19.970245 19263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:19.970271 19263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:19.971551 19263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:19.971596 19263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:19.971623 19263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:19.973184 19263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:39673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:19.973381 19263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:19.973567 19263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:19.979103 19273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:19.979140 19275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:19.979108 19263 server_base.cc:1034] running on GCE node
W20260808 07:04:19.979156 19272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:19.979629 19263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:19.979852 19263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:19.981001 19263 hybrid_clock.cc:648] HybridClock initialized: now 1786172659980971 us; error 48 us; skew 500 ppm
Aug 08 07:04:19 dist-test-slave-nfb5 krb5kdc[18729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:19.983549 19263 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:04:19.984017 19263 webserver.cc:460] Webserver started at http://127.24.181.67:33931/ using document root <none> and password file <none>
I20260808 07:04:19.984254 19263 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:19.984301 19263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:19.984443 19263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:19.985598 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "0471a847db504d718a1fb512a822244e"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "c20708851699421d8df538653b5d0dea"
server_key_iv: "f0545ea73567abd7928d4578a1134ebc"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.985986 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "0471a847db504d718a1fb512a822244e"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "c20708851699421d8df538653b5d0dea"
server_key_iv: "f0545ea73567abd7928d4578a1134ebc"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.987389 19263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:19.988153 19282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:19.988359 19263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:19.988445 19263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "0471a847db504d718a1fb512a822244e"
format_stamp: "Formatted at 2026-08-08 07:04:19 on dist-test-slave-nfb5"
server_key: "c20708851699421d8df538653b5d0dea"
server_key_iv: "f0545ea73567abd7928d4578a1134ebc"
server_key_version: "encryptionkey@0"
I20260808 07:04:19.988500 19263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:20.008229 19263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:20.010375 19263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:20.010489 19263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:20.010689 19263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:20.011072 19263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:20.011111 19263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.011134 19263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:20.011181 19263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.017239 19263 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:36055
I20260808 07:04:20.017325 19438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:36055 every 8 connection(s)
I20260808 07:04:20.017637 19263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:20.023638 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 19263
I20260808 07:04:20.023766 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:20.023931 25301 external_mini_cluster.cc:1428] Setting key e82d22af3cb36837a7df124f117727c0
Aug 08 07:04:20 dist-test-slave-nfb5 krb5kdc[18729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:20.027829 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.018540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:52167 (local address 127.24.181.126:42501)
0808 07:04:20.018626 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.018628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.018805 (+ 177us) server_negotiation.cc:407] Connection header received
0808 07:04:20.019366 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.019367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.019378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.019442 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.020363 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.020888 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.021925 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.022110 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.022169 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.024763 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.024769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.024769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:20.024776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.026302 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:20.026726 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:20.026727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:20.026727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:20.026788 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:20.027085 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:20.027086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:20.027086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:20.027256 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.027349 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.027635 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.027736 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:04:20.027979 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.018479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:42501 (local address 127.24.181.67:52167)
0808 07:04:20.018660 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.018703 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.019270 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.019496 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.019500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.020081 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:20.020273 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.020278 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.020975 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.020976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.021765 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.021769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.022197 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.022198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.022316 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.022865 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:20.022874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:20.024654 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:20.026408 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:20.026416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:20.026419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:20.026647 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:20.026859 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:20.026859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:20.026860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:20.027009 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:20.027341 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:20.027348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:20.027491 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.027588 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.027749 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260808 07:04:20.028506 19439 heartbeater.cc:347] Connected to a master server at 127.24.181.126:42501
I20260808 07:04:20.028592 19439 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:20.028777 19439 heartbeater.cc:511] Master 127.24.181.126:42501 requested a full tablet report, sending...
I20260808 07:04:20.029117 18796 ts_manager.cc:194] Registered new tserver with Master: 0471a847db504d718a1fb512a822244e (127.24.181.67:36055)
I20260808 07:04:20.029510 18796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:52167
I20260808 07:04:20.035905 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:20.044221 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.037570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.24.181.126:42501)
0808 07:04:20.037628 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.037630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.037755 (+ 125us) server_negotiation.cc:407] Connection header received
0808 07:04:20.037794 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.037795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.037804 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.037848 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.038579 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.039287 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.040038 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.040225 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.040283 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.041613 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.041614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.041614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:20.041621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.043083 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:20.043444 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:20.043445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:20.043446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:20.043499 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:20.043720 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:20.043721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:20.043721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:20.043846 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.043926 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.044046 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.044095 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:04:20.046053 18796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:20.047801 18796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:20.055797 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.051147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.24.181.67:36055)
0808 07:04:20.051382 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.051388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.051987 (+ 599us) server_negotiation.cc:407] Connection header received
0808 07:04:20.052073 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.052075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.052090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.052172 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.052274 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.052915 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.054392 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055340 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055429 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055492 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.055632 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.055697 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":100,"threads_started":1}
I20260808 07:04:20.055797 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.051031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:36055 (local address 127.0.0.1:60878)
0808 07:04:20.051907 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.051917 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.051992 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.052170 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.052171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.052173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.052217 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.052219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.053020 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.053021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.054288 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.054291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055432 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.055514 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055527 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.055602 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.055677 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
I20260808 07:04:20.055771 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.050252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42207 (local address 127.0.0.1:34686)
0808 07:04:20.051447 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.051455 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.051494 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.052100 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.052101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.052103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.052136 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.052137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.053395 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.053396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.054580 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.054582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055554 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.055617 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055620 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.055665 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.055694 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"thread_start_us":597,"threads_started":1}
I20260808 07:04:20.055996 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.051213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.24.181.65:41593)
0808 07:04:20.051614 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.051636 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.051648 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:20.051690 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.051692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.051709 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.051794 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.051957 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.053113 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.054464 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055340 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055429 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055492 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.055822 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.055874 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":174,"threads_started":1}
I20260808 07:04:20.056049 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.050211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41593 (local address 127.0.0.1:40786)
0808 07:04:20.051214 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.051258 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.051328 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.051797 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.051800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.051802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.051882 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.051883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.053215 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.053215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.054361 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.054380 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055537 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.055617 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055622 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.055683 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.055726 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":631,"threads_started":1}
I20260808 07:04:20.056241 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.051191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.24.181.66:42207)
0808 07:04:20.051616 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.051622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.051632 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:20.051671 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.051673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.051688 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.052009 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.052253 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.053286 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.054658 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.055380 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.055464 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.055521 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.055851 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.056174 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":205,"threads_started":1}
I20260808 07:04:20.056967 19164 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e22ec031453dbd36f61960fb0780 (DEFAULT_TABLE table=test-table [id=365799592e2c424faf4c73ec59b3319c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.057086 18966 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e22ec031453dbd36f61960fb0780 (DEFAULT_TABLE table=test-table [id=365799592e2c424faf4c73ec59b3319c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.057374 19164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e22ec031453dbd36f61960fb0780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.057467 18966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e22ec031453dbd36f61960fb0780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.060361 19467 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Bootstrap starting.
I20260808 07:04:20.060457 19468 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Bootstrap starting.
I20260808 07:04:20.061030 19468 tablet_bootstrap.cc:654] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.060951 19350 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e22ec031453dbd36f61960fb0780 (DEFAULT_TABLE table=test-table [id=365799592e2c424faf4c73ec59b3319c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.061126 19467 tablet_bootstrap.cc:654] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.061313 19350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e22ec031453dbd36f61960fb0780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.061606 19468 log.cc:824] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.061807 19467 log.cc:824] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.062485 19468 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: No bootstrap required, opened a new log
I20260808 07:04:20.062528 19467 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: No bootstrap required, opened a new log
I20260808 07:04:20.062587 19468 ts_tablet_manager.cc:1397] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:20.062609 19467 ts_tablet_manager.cc:1397] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:20.064200 19467 raft_consensus.cc:348] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.064363 19467 raft_consensus.cc:374] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.064409 19467 raft_consensus.cc:729] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49829d49c7ed46dd819e07b55fe808fc, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.064527 19467 consensus_queue.cc:260] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.064836 19063 heartbeater.cc:503] Master 127.24.181.126:42501 was elected leader, sending a full tablet report...
I20260808 07:04:20.065037 19468 raft_consensus.cc:348] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.065297 19472 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Bootstrap starting.
I20260808 07:04:20.065332 19468 raft_consensus.cc:374] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.065375 19468 raft_consensus.cc:729] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df0f303e8474f8a9bc702a276ef6f5a, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.065503 19468 consensus_queue.cc:260] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.064826 19467 ts_tablet_manager.cc:1428] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:20.065884 19468 ts_tablet_manager.cc:1428] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:04:20.065913 19257 heartbeater.cc:503] Master 127.24.181.126:42501 was elected leader, sending a full tablet report...
I20260808 07:04:20.066095 19472 tablet_bootstrap.cc:654] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.066710 19472 log.cc:824] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.067625 19472 tablet_bootstrap.cc:492] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: No bootstrap required, opened a new log
I20260808 07:04:20.067721 19472 ts_tablet_manager.cc:1397] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:04:20.069072 19472 raft_consensus.cc:348] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.069242 19472 raft_consensus.cc:374] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.069267 19472 raft_consensus.cc:729] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0471a847db504d718a1fb512a822244e, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.069387 19472 consensus_queue.cc:260] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.069655 19472 ts_tablet_manager.cc:1428] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:04:20.069821 19439 heartbeater.cc:503] Master 127.24.181.126:42501 was elected leader, sending a full tablet report...
W20260808 07:04:20.094385 19258 tablet.cc:2369] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:20.164501 19066 tablet.cc:2369] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:20.191995 19477 raft_consensus.cc:482] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:20.192130 19477 raft_consensus.cc:504] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.192610 19477 leader_election.cc:288] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49829d49c7ed46dd819e07b55fe808fc (127.24.181.65:41593), 4df0f303e8474f8a9bc702a276ef6f5a (127.24.181.66:42207)
I20260808 07:04:20.196668 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.193017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:57499 (local address 127.24.181.66:42207)
0808 07:04:20.193063 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.193065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.193122 (+ 57us) server_negotiation.cc:407] Connection header received
0808 07:04:20.193181 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.193182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.193192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.193242 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.193418 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.194185 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.195803 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.196282 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.196372 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.196381 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.196550 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.196591 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:20.196666 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.192889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:57181 (local address 127.24.181.65:41593)
0808 07:04:20.192940 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.192942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.193002 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:04:20.193120 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.193121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.193133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.193202 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.193418 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.194174 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.195856 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.196256 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.196364 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.196374 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.196544 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.196585 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:04:20.196708 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.192831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41593 (local address 127.24.181.67:57181)
0808 07:04:20.192924 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.192936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.193005 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.193235 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.193236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.193238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.193337 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.193339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.194270 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.194271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.195719 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.195720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.196368 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.196369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.196458 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.196461 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.196533 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.196580 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":101}
I20260808 07:04:20.196908 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.192841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:42207 (local address 127.24.181.67:57499)
0808 07:04:20.193052 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.193062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.193117 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.193252 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.193252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.193253 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.193343 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.193344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.194270 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.194271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.195709 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.195713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.196388 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.196389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.196476 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.196479 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.196541 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.196584 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":79,"thread_start_us":149,"threads_started":1}
I20260808 07:04:20.197094 19000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e22ec031453dbd36f61960fb0780" candidate_uuid: "0471a847db504d718a1fb512a822244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49829d49c7ed46dd819e07b55fe808fc" is_pre_election: true
I20260808 07:04:20.197279 19000 raft_consensus.cc:2454] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0471a847db504d718a1fb512a822244e in term 0.
I20260808 07:04:20.197255 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e22ec031453dbd36f61960fb0780" candidate_uuid: "0471a847db504d718a1fb512a822244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" is_pre_election: true
I20260808 07:04:20.197381 19190 raft_consensus.cc:2454] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0471a847db504d718a1fb512a822244e in term 0.
I20260808 07:04:20.197592 19284 leader_election.cc:302] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0471a847db504d718a1fb512a822244e, 49829d49c7ed46dd819e07b55fe808fc; no voters:
I20260808 07:04:20.197731 19477 raft_consensus.cc:2781] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:20.197815 19477 raft_consensus.cc:482] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:20.197860 19477 raft_consensus.cc:3037] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.198599 19477 raft_consensus.cc:504] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.198755 19477 leader_election.cc:288] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [CANDIDATE]: Term 1 election: Requested vote from peers 49829d49c7ed46dd819e07b55fe808fc (127.24.181.65:41593), 4df0f303e8474f8a9bc702a276ef6f5a (127.24.181.66:42207)
I20260808 07:04:20.198943 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e22ec031453dbd36f61960fb0780" candidate_uuid: "0471a847db504d718a1fb512a822244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df0f303e8474f8a9bc702a276ef6f5a"
I20260808 07:04:20.198976 19000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e22ec031453dbd36f61960fb0780" candidate_uuid: "0471a847db504d718a1fb512a822244e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49829d49c7ed46dd819e07b55fe808fc"
I20260808 07:04:20.199026 19190 raft_consensus.cc:3037] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.199092 19000 raft_consensus.cc:3037] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.199834 19190 raft_consensus.cc:2454] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0471a847db504d718a1fb512a822244e in term 1.
I20260808 07:04:20.199898 19000 raft_consensus.cc:2454] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0471a847db504d718a1fb512a822244e in term 1.
I20260808 07:04:20.200052 19283 leader_election.cc:302] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0471a847db504d718a1fb512a822244e, 4df0f303e8474f8a9bc702a276ef6f5a; no voters:
I20260808 07:04:20.200138 19477 raft_consensus.cc:2781] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:20.200327 19477 raft_consensus.cc:686] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 1 LEADER]: Becoming Leader. State: Replica: 0471a847db504d718a1fb512a822244e, State: Running, Role: LEADER
I20260808 07:04:20.200438 19477 consensus_queue.cc:237] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } }
I20260808 07:04:20.201375 18796 catalog_manager.cc:5495] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0471a847db504d718a1fb512a822244e (127.24.181.67). New cstate: current_term: 1 leader_uuid: "0471a847db504d718a1fb512a822244e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0471a847db504d718a1fb512a822244e" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 36055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 } health_report { overall_health: UNKNOWN } } }
I20260808 07:04:20.236428 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.233968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.24.181.67:36055)
0808 07:04:20.234042 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.234044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.234054 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:20.234098 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.234099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.234119 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.234193 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:20.234357 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.234849 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.235625 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.235826 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.235897 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.236073 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:20.236160 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:20.236270 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.236301 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.236312 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:04:20.238408 19000 raft_consensus.cc:1261] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 1 FOLLOWER]: Refusing update from remote peer 0471a847db504d718a1fb512a822244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:20.238408 19190 raft_consensus.cc:1261] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 1 FOLLOWER]: Refusing update from remote peer 0471a847db504d718a1fb512a822244e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:20.238806 19481 consensus_queue.cc:1035] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "49829d49c7ed46dd819e07b55fe808fc" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:20.238962 19477 consensus_queue.cc:1035] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df0f303e8474f8a9bc702a276ef6f5a" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 42207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:20.241227 19486 mvcc.cc:205] Tried to move back new op lower bound from 7316163216333623296 to 7316163216181530624. Current Snapshot: MvccSnapshot[applied={T|T < 7316163216333623296}]
I20260808 07:04:20.241556 19487 mvcc.cc:205] Tried to move back new op lower bound from 7316163216333623296 to 7316163216181530624. Current Snapshot: MvccSnapshot[applied={T|T < 7316163216333623296}]
I20260808 07:04:20.243745 18796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:20.243849 18796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:20.244609 18796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ba8023aa4354f03804905aaaa9a647f: Sending DeleteTablet for 3 replicas of tablet 6632e22ec031453dbd36f61960fb0780
I20260808 07:04:20.244979 19164 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e22ec031453dbd36f61960fb0780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:34686
I20260808 07:04:20.244999 18966 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e22ec031453dbd36f61960fb0780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:40786
I20260808 07:04:20.245256 19499 tablet_replica.cc:330] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc: stopping tablet replica
I20260808 07:04:20.245282 19497 tablet_replica.cc:330] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: stopping tablet replica
I20260808 07:04:20.245390 19497 raft_consensus.cc:2229] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:20.245414 19499 raft_consensus.cc:2229] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:20.245396 19350 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e22ec031453dbd36f61960fb0780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:60878
I20260808 07:04:20.245539 19499 raft_consensus.cc:2258] T 6632e22ec031453dbd36f61960fb0780 P 49829d49c7ed46dd819e07b55fe808fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:20.245599 19497 raft_consensus.cc:2258] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:20.245761 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 18886
I20260808 07:04:20.246130 19497 ts_tablet_manager.cc:1905] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:20.247937 19497 ts_tablet_manager.cc:1918] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:20.248006 19497 log.cc:1196] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/6632e22ec031453dbd36f61960fb0780
I20260808 07:04:20.248310 19497 ts_tablet_manager.cc:1939] T 6632e22ec031453dbd36f61960fb0780 P 4df0f303e8474f8a9bc702a276ef6f5a: Deleting consensus metadata
I20260808 07:04:20.251757 18775 catalog_manager.cc:4841] TS 4df0f303e8474f8a9bc702a276ef6f5a (127.24.181.66:42207): tablet 6632e22ec031453dbd36f61960fb0780 (table test-table [id=365799592e2c424faf4c73ec59b3319c]) successfully deleted
I20260808 07:04:20.252007 19500 tablet_replica.cc:330] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: stopping tablet replica
I20260808 07:04:20.252120 19500 raft_consensus.cc:2229] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:20.252230 19500 raft_consensus.cc:2258] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:20.252707 19500 ts_tablet_manager.cc:1905] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:20.254808 19500 ts_tablet_manager.cc:1918] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:20.254891 19500 log.cc:1196] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/6632e22ec031453dbd36f61960fb0780
I20260808 07:04:20.255246 19500 ts_tablet_manager.cc:1939] T 6632e22ec031453dbd36f61960fb0780 P 0471a847db504d718a1fb512a822244e: Deleting consensus metadata
I20260808 07:04:20.255693 18777 catalog_manager.cc:4841] TS 0471a847db504d718a1fb512a822244e (127.24.181.67:36055): tablet 6632e22ec031453dbd36f61960fb0780 (table test-table [id=365799592e2c424faf4c73ec59b3319c]) successfully deleted
W20260808 07:04:20.257918 18777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:41593 (error 108)
I20260808 07:04:20.258492 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.258349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:41593 (local address 127.0.0.1:40790)
0808 07:04:20.258408 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.258450 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:41593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:04:20.258671 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 19071
W20260808 07:04:20.258667 18777 catalog_manager.cc:4584] TS 49829d49c7ed46dd819e07b55fe808fc (127.24.181.65:41593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6632e22ec031453dbd36f61960fb0780: Network error: Client connection negotiation failed: client connection to 127.24.181.65:41593: connect: Connection refused (error 111)
I20260808 07:04:20.267477 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 19263
I20260808 07:04:20.280865 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 18752
2026-08-08T07:04:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2302 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T07:04:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:20Z Disabled control of system clock
I20260808 07:04:20.312779 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44429
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:20.394779 19508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:20.395026 19508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:20.395049 19508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:20.396193 19508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:20.396237 19508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:20.396250 19508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:20.396263 19508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:20.396303 19508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:20.397727 19508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:20.397990 19508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:20.398294 19508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:20.403359 19514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:20.403378 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.403445 19508 server_base.cc:1034] running on GCE node
W20260808 07:04:20.403368 19518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.403824 19508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:20.404172 19508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:20.405328 19508 hybrid_clock.cc:648] HybridClock initialized: now 1786172660405298 us; error 44 us; skew 500 ppm
I20260808 07:04:20.406770 19508 webserver.cc:460] Webserver started at http://127.24.181.126:36731/ using document root <none> and password file <none>
I20260808 07:04:20.407095 19508 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:20.407150 19508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:20.407357 19508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:20.408545 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "bb6fbc57bc3540d6b3f202ad283dc574"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "2c1825a0e04c542f0ec18f8889e809b7"
server_key_iv: "06cc3836b68981ec1ef7d6687c51e2f2"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.408944 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "bb6fbc57bc3540d6b3f202ad283dc574"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "2c1825a0e04c542f0ec18f8889e809b7"
server_key_iv: "06cc3836b68981ec1ef7d6687c51e2f2"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.410382 19508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:20.411325 19525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.411497 19508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.411573 19508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "bb6fbc57bc3540d6b3f202ad283dc574"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "2c1825a0e04c542f0ec18f8889e809b7"
server_key_iv: "06cc3836b68981ec1ef7d6687c51e2f2"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.411630 19508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:20.426399 19508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:20.426710 19508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:20.426820 19508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:20.431519 19508 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44429
I20260808 07:04:20.431594 19601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44429 every 8 connection(s)
I20260808 07:04:20.431993 19508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:20.433092 19603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.435554 19603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: Bootstrap starting.
I20260808 07:04:20.436127 19603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.436383 19603 log.cc:824] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.437063 19603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: No bootstrap required, opened a new log
I20260808 07:04:20.437484 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 19508
I20260808 07:04:20.437609 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:20.437816 25301 external_mini_cluster.cc:1428] Setting key 06320f8aca667e0524eba5a2a3c2239d
I20260808 07:04:20.439278 19603 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } }
I20260808 07:04:20.439450 19603 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.439489 19603 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6fbc57bc3540d6b3f202ad283dc574, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.439631 19603 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } }
I20260808 07:04:20.439738 19603 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:20.439782 19603 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:20.439837 19603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.440426 19603 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } }
I20260808 07:04:20.440533 19603 leader_election.cc:302] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb6fbc57bc3540d6b3f202ad283dc574; no voters:
I20260808 07:04:20.440783 19603 leader_election.cc:288] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:20.440826 19610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:20.440949 19610 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [term 1 LEADER]: Becoming Leader. State: Replica: bb6fbc57bc3540d6b3f202ad283dc574, State: Running, Role: LEADER
I20260808 07:04:20.441039 19610 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } }
I20260808 07:04:20.441077 19603 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:20.441391 19610 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb6fbc57bc3540d6b3f202ad283dc574. Latest consensus state: current_term: 1 leader_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } } }
I20260808 07:04:20.441468 19610 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:20.441670 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6fbc57bc3540d6b3f202ad283dc574" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44429 } } }
I20260808 07:04:20.441738 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:20.441699 19616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:20.442406 19616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:20.444358 19635 mvcc.cc:205] Tried to move back new op lower bound from 7316163217176305664 to 7316163217166929920. Current Snapshot: MvccSnapshot[applied={T|T < 7316163217176305664}]
I20260808 07:04:20.444795 19616 catalog_manager.cc:1347] Generated new cluster ID: 97be4ef52c8e404abca0f97da5df855a
I20260808 07:04:20.444852 19616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:20.445343 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.438416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.24.181.126:44429)
0808 07:04:20.438620 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.438627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.438653 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:04:20.439252 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.439259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.439323 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.439453 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.439696 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.441703 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.442906 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.443525 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.443657 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.444472 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.444476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.444477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:20.444678 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.444735 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:20.444751 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.444818 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.445043 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.445127 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260808 07:04:20.451347 19616 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:20.451817 19616 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:20.454581 19616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: Generated new TSK 0
I20260808 07:04:20.454763 19616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:20.461283 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--builtin_ntp_servers=127.24.181.84:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:20.546576 19637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:20.546777 19637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:20.546799 19637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:20.547974 19637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:20.548017 19637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:20.548043 19637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:20.549396 19637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:20.549669 19637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:20.549969 19637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:20.556032 19645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.556048 19637 server_base.cc:1034] running on GCE node
W20260808 07:04:20.556143 19644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:20.556027 19650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.556576 19637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:20.556850 19637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:20.558003 19637 hybrid_clock.cc:648] HybridClock initialized: now 1786172660557989 us; error 33 us; skew 500 ppm
I20260808 07:04:20.559360 19637 webserver.cc:460] Webserver started at http://127.24.181.65:33767/ using document root <none> and password file <none>
I20260808 07:04:20.559639 19637 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:20.559693 19637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:20.559839 19637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:20.560920 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "6b6bf43ea3b94e399a4e6481e284c174"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a2c2e540e4e0d2fc62ac1b0fb028a607"
server_key_iv: "4711b27fd89f81e8fe775fc39034c790"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.561286 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "6b6bf43ea3b94e399a4e6481e284c174"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a2c2e540e4e0d2fc62ac1b0fb028a607"
server_key_iv: "4711b27fd89f81e8fe775fc39034c790"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.562654 19637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:20.563491 19659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.563643 19637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.563755 19637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "6b6bf43ea3b94e399a4e6481e284c174"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a2c2e540e4e0d2fc62ac1b0fb028a607"
server_key_iv: "4711b27fd89f81e8fe775fc39034c790"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.563832 19637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:20.577870 19637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:20.578161 19637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:20.578299 19637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:20.578523 19637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:20.578843 19637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:20.578879 19637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.578900 19637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:20.578953 19637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.585177 19637 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:39033
I20260808 07:04:20.585237 19822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:39033 every 8 connection(s)
I20260808 07:04:20.585578 19637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:20.586858 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 19637
I20260808 07:04:20.587009 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:20.587255 25301 external_mini_cluster.cc:1428] Setting key 88e8cf6acecaf8d6488631259a028c2d
I20260808 07:04:20.588339 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--builtin_ntp_servers=127.24.181.84:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:04:20.591724 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.586361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:40523 (local address 127.24.181.126:44429)
0808 07:04:20.586428 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.586430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.586688 (+ 258us) server_negotiation.cc:407] Connection header received
0808 07:04:20.587363 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.587365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.587381 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.587435 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.588262 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.588866 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.589933 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.590249 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.590312 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.591433 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.591435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.591435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:20.591439 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.591453 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:20.591458 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.591523 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.591610 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.591638 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:04:20.591972 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.586349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44429 (local address 127.24.181.65:40523)
0808 07:04:20.586570 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.586602 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.587235 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.587504 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.587513 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.587521 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:20.588164 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.588169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.588961 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.588962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.589801 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.589804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.590409 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.590410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.590574 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.591285 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:20.591286 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:20.591313 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:20.591335 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:20.591526 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:20.591531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:20.591535 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.591612 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.591711 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260808 07:04:20.592625 19827 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44429
I20260808 07:04:20.592751 19827 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:20.592983 19827 heartbeater.cc:511] Master 127.24.181.126:44429 requested a full tablet report, sending...
I20260808 07:04:20.593519 19553 ts_manager.cc:194] Registered new tserver with Master: 6b6bf43ea3b94e399a4e6481e284c174 (127.24.181.65:39033)
I20260808 07:04:20.594142 19553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.65:40523
W20260808 07:04:20.679121 19830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:20.679319 19830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:20.679342 19830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:20.680465 19830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:20.680505 19830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:20.680554 19830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:20.681890 19830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:20.682128 19830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:20.682410 19830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:20.687402 19830 server_base.cc:1034] running on GCE node
W20260808 07:04:20.687361 19836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:20.687373 19839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:20.687371 19837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.687728 19830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:20.687991 19830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:20.689155 19830 hybrid_clock.cc:648] HybridClock initialized: now 1786172660689151 us; error 37 us; skew 500 ppm
I20260808 07:04:20.693116 19830 webserver.cc:460] Webserver started at http://127.24.181.66:45457/ using document root <none> and password file <none>
I20260808 07:04:20.693384 19830 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:20.693455 19830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:20.693599 19830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:20.694619 19830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "3c98ca8862984f71a150782aeeefea78"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "c2fc3c683a3e5c2152e14080631d384e"
server_key_iv: "39a535534605eb1d85bd97ce7ae39f2e"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.695044 19830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "3c98ca8862984f71a150782aeeefea78"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "c2fc3c683a3e5c2152e14080631d384e"
server_key_iv: "39a535534605eb1d85bd97ce7ae39f2e"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.696339 19830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:20.697156 19845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.697325 19830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:20.697422 19830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "3c98ca8862984f71a150782aeeefea78"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "c2fc3c683a3e5c2152e14080631d384e"
server_key_iv: "39a535534605eb1d85bd97ce7ae39f2e"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.697501 19830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:20.713738 19830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:20.714020 19830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:20.714161 19830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:20.714391 19830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:20.714722 19830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:20.714788 19830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.714844 19830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:20.714872 19830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.720899 19830 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:36851
I20260808 07:04:20.720957 20002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:36851 every 8 connection(s)
I20260808 07:04:20.721280 19830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:20.726104 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.721972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:46143 (local address 127.24.181.126:44429)
0808 07:04:20.722065 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.722067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.722271 (+ 204us) server_negotiation.cc:407] Connection header received
0808 07:04:20.722742 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.722744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.722754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.722805 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.723471 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.723969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.724914 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.725096 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.725152 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.725884 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.725885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.725886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:20.725890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.725903 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:20.725907 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.725955 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.726020 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.726045 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:04:20.726269 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.721934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44429 (local address 127.24.181.66:46143)
0808 07:04:20.722150 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.722183 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.722655 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.722867 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.722870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.722878 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:20.723395 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.723400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.724055 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.724056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.724840 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.724842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.725169 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.725170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.725280 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.725761 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:20.725762 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:20.725788 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:20.725809 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:20.725957 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:20.725959 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:20.725962 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.726007 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.726078 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260808 07:04:20.726773 20003 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44429
I20260808 07:04:20.726876 20003 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:20.727075 20003 heartbeater.cc:511] Master 127.24.181.126:44429 requested a full tablet report, sending...
I20260808 07:04:20.727403 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 19830
I20260808 07:04:20.727514 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:20.727732 25301 external_mini_cluster.cc:1428] Setting key e8d616421014760b78cb6aaa49371264
I20260808 07:04:20.728140 19553 ts_manager.cc:194] Registered new tserver with Master: 3c98ca8862984f71a150782aeeefea78 (127.24.181.66:36851)
I20260808 07:04:20.728639 19553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.66:46143
I20260808 07:04:20.728909 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--builtin_ntp_servers=127.24.181.84:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:20.817684 20009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:20.817974 20009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:20.818015 20009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:20.820088 20009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:20.820155 20009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:20.820214 20009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:20.822484 20009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:44429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:20.822764 20009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:20.823139 20009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:20.829934 20017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:20.829931 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.830216 20009 server_base.cc:1034] running on GCE node
W20260808 07:04:20.830327 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:20.830550 20009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:20.830806 20009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:20.832864 20009 hybrid_clock.cc:648] HybridClock initialized: now 1786172660832851 us; error 55 us; skew 500 ppm
I20260808 07:04:20.834244 20009 webserver.cc:460] Webserver started at http://127.24.181.67:34023/ using document root <none> and password file <none>
I20260808 07:04:20.834523 20009 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:20.834602 20009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:20.834750 20009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:20.836262 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "c3363222887243e6b280bae428200dc1"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a608841ae44211f1133832f66833dd3d"
server_key_iv: "ffe0e62a2547c8c0364b2ac70f522f8d"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.836709 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "c3363222887243e6b280bae428200dc1"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a608841ae44211f1133832f66833dd3d"
server_key_iv: "ffe0e62a2547c8c0364b2ac70f522f8d"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.838615 20009 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:04:20.839669 20030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.839892 20009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:20.839996 20009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "c3363222887243e6b280bae428200dc1"
format_stamp: "Formatted at 2026-08-08 07:04:20 on dist-test-slave-nfb5"
server_key: "a608841ae44211f1133832f66833dd3d"
server_key_iv: "ffe0e62a2547c8c0364b2ac70f522f8d"
server_key_version: "encryptionkey@0"
I20260808 07:04:20.840080 20009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:20.863155 20009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:20.863554 20009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:20.863729 20009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:20.864055 20009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:20.864504 20009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:20.864563 20009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.864599 20009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:20.864630 20009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:20.871294 20009 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:40323
I20260808 07:04:20.871346 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:40323 every 8 connection(s)
I20260808 07:04:20.871678 20009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:20.874215 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 20009
I20260808 07:04:20.874328 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:20.874545 25301 external_mini_cluster.cc:1428] Setting key 8c22ae30ce683bdb391218dc4219f717
I20260808 07:04:20.877003 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.872383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:53333 (local address 127.24.181.126:44429)
0808 07:04:20.872431 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.872433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.872661 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:04:20.873141 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.873142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.873154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.873222 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.873843 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.874591 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.875826 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.876008 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.876062 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.876796 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.876797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.876798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:20.876802 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.876815 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:20.876819 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.876861 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.876918 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.876943 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:04:20.877208 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.872360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:44429 (local address 127.24.181.67:53333)
0808 07:04:20.872535 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.872568 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.873059 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.873271 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.873274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.873283 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:20.873770 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.873774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.874682 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.874684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.875753 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.875756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.876082 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.876083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.876193 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.876676 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:20.876677 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:20.876703 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:20.876726 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:20.876861 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:20.876863 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:20.876865 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.876911 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.876983 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260808 07:04:20.877648 20211 heartbeater.cc:347] Connected to a master server at 127.24.181.126:44429
I20260808 07:04:20.877723 20211 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:20.877924 20211 heartbeater.cc:511] Master 127.24.181.126:44429 requested a full tablet report, sending...
I20260808 07:04:20.878283 19553 ts_manager.cc:194] Registered new tserver with Master: c3363222887243e6b280bae428200dc1 (127.24.181.67:40323)
I20260808 07:04:20.878641 19553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.67:53333
I20260808 07:04:20.886215 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:20.890506 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.887680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.24.181.126:44429)
0808 07:04:20.887798 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.887800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.887853 (+ 53us) server_negotiation.cc:407] Connection header received
0808 07:04:20.887897 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.887898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.887909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.887956 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:20.888118 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.888568 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.889281 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.889450 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.889498 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.890203 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:20.890203 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:20.890204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:20.890207 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:20.890218 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:20.890222 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:20.890285 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.890363 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.890374 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 07:04:20.892082 19553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:20.893682 19553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:20.899956 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.895858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36851 (local address 127.0.0.1:54914)
0808 07:04:20.895982 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.896021 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.896080 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.896390 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.896393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.896395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.896476 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.896478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.897784 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.897786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.898694 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.898697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.899665 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.899746 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.899758 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.899832 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.899880 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260808 07:04:20.899976 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.895956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.24.181.66:36851)
0808 07:04:20.896121 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.896126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.896136 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:20.896193 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.896196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.896213 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.896306 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.896570 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.897672 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.898800 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.899573 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899655 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.899758 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.899844 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.899887 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260808 07:04:20.900418 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.896083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:40323 (local address 127.0.0.1:34492)
0808 07:04:20.896231 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.896244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.896288 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.896904 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.896905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.896907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.896954 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.896956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.897811 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.897812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.899007 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.899010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.899989 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.900240 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.900246 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.900292 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.900331 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":23,"thread_start_us":87,"threads_started":1}
I20260808 07:04:20.900650 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.896405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.24.181.65:39033)
0808 07:04:20.896552 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.896556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.896813 (+ 257us) server_negotiation.cc:407] Connection header received
0808 07:04:20.896961 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.896964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.896981 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.897057 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.897247 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.898333 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899455 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.900186 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.900257 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.900333 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.900516 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.900556 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260808 07:04:20.900739 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.896497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.24.181.67:40323)
0808 07:04:20.896675 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.896679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.896689 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:20.896740 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.896742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.896759 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.896835 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.897019 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.897712 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899093 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.899803 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.899872 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.899950 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.900570 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.900615 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260808 07:04:20.900601 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.896279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:39033 (local address 127.0.0.1:59792)
0808 07:04:20.896742 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.896751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.896843 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.897125 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.897126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.897128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.897167 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.897168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.898431 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.898431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.899367 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.899368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.900365 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.900365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.900410 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.900413 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.900495 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.900534 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":68,"threads_started":1}
I20260808 07:04:20.901036 19905 tablet_service.cc:1467] Processing CreateTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 (DEFAULT_TABLE table=test-table [id=f2c1edee3d5a4edd8a5533ff8683ffef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.901417 19905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa375d6aa05f4c8890879b26ab14cba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.903596 19728 tablet_service.cc:1467] Processing CreateTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 (DEFAULT_TABLE table=test-table [id=f2c1edee3d5a4edd8a5533ff8683ffef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.903688 20104 tablet_service.cc:1467] Processing CreateTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 (DEFAULT_TABLE table=test-table [id=f2c1edee3d5a4edd8a5533ff8683ffef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:20.903947 19728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa375d6aa05f4c8890879b26ab14cba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.904042 20104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa375d6aa05f4c8890879b26ab14cba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:20.904464 20230 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Bootstrap starting.
I20260808 07:04:20.905236 20230 tablet_bootstrap.cc:654] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.905697 20230 log.cc:824] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.906597 20230 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: No bootstrap required, opened a new log
I20260808 07:04:20.906755 20230 ts_tablet_manager.cc:1397] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:20.906828 20232 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Bootstrap starting.
I20260808 07:04:20.907495 20232 tablet_bootstrap.cc:654] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.907773 20233 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: Bootstrap starting.
I20260808 07:04:20.907869 20232 log.cc:824] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.908506 20233 tablet_bootstrap.cc:654] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:20.908690 20232 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: No bootstrap required, opened a new log
I20260808 07:04:20.908774 20232 ts_tablet_manager.cc:1397] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:20.908849 20233 log.cc:824] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:20.909554 20230 raft_consensus.cc:348] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.909672 20233 tablet_bootstrap.cc:492] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: No bootstrap required, opened a new log
I20260808 07:04:20.909703 20230 raft_consensus.cc:374] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.909739 20233 ts_tablet_manager.cc:1397] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:20.909746 20230 raft_consensus.cc:729] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c98ca8862984f71a150782aeeefea78, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.909885 20230 consensus_queue.cc:260] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.910176 20003 heartbeater.cc:503] Master 127.24.181.126:44429 was elected leader, sending a full tablet report...
I20260808 07:04:20.910158 20230 ts_tablet_manager.cc:1428] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:20.911542 20232 raft_consensus.cc:348] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.911720 20232 raft_consensus.cc:374] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.911770 20232 raft_consensus.cc:729] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3363222887243e6b280bae428200dc1, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.911891 20232 consensus_queue.cc:260] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.912140 20232 ts_tablet_manager.cc:1428] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:04:20.912178 20211 heartbeater.cc:503] Master 127.24.181.126:44429 was elected leader, sending a full tablet report...
I20260808 07:04:20.912541 20233 raft_consensus.cc:348] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.912707 20233 raft_consensus.cc:374] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:20.912743 20233 raft_consensus.cc:729] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6bf43ea3b94e399a4e6481e284c174, State: Initialized, Role: FOLLOWER
I20260808 07:04:20.912858 20233 consensus_queue.cc:260] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.913108 20233 ts_tablet_manager.cc:1428] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:04:20.913228 19827 heartbeater.cc:503] Master 127.24.181.126:44429 was elected leader, sending a full tablet report...
I20260808 07:04:20.920148 20240 raft_consensus.cc:482] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:20.920253 20240 raft_consensus.cc:504] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.920564 20240 leader_election.cc:288] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c98ca8862984f71a150782aeeefea78 (127.24.181.66:36851), 6b6bf43ea3b94e399a4e6481e284c174 (127.24.181.65:39033)
I20260808 07:04:20.923404 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.920767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:46045 (local address 127.24.181.66:36851)
0808 07:04:20.920806 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.920808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.920816 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:20.920878 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.920879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.920890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.920950 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.921171 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.921698 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.922640 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.923073 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.923131 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.923153 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.923292 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.923336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:04:20.923403 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.920712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:36851 (local address 127.24.181.67:46045)
0808 07:04:20.920768 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.920780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.920821 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.920989 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.920990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.920992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.921092 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.921094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.921781 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.921782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.922555 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.922557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.923145 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.923146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.923218 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.923221 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.923273 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.923310 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":30}
I20260808 07:04:20.923488 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.920899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:44243 (local address 127.24.181.65:39033)
0808 07:04:20.920945 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.920946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.921061 (+ 115us) server_negotiation.cc:407] Connection header received
0808 07:04:20.921104 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.921105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.921117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.921177 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:20.921305 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.921764 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.922684 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.923088 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.923173 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.923180 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.923373 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.923413 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:20.923661 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.920841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:39033 (local address 127.24.181.67:44243)
0808 07:04:20.920993 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.921003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:20.921060 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:20.921181 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:20.921182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:20.921183 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:20.921246 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.921247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.921845 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.921845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.922582 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:20.922583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.923184 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.923185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:20.923290 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.923294 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:20.923359 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:04:20.923396 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260808 07:04:20.923748 19934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa375d6aa05f4c8890879b26ab14cba6" candidate_uuid: "c3363222887243e6b280bae428200dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c98ca8862984f71a150782aeeefea78" is_pre_election: true
I20260808 07:04:20.923882 19934 raft_consensus.cc:2454] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3363222887243e6b280bae428200dc1 in term 0.
I20260808 07:04:20.924083 20034 leader_election.cc:302] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c98ca8862984f71a150782aeeefea78, c3363222887243e6b280bae428200dc1; no voters:
I20260808 07:04:20.924127 19754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa375d6aa05f4c8890879b26ab14cba6" candidate_uuid: "c3363222887243e6b280bae428200dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6bf43ea3b94e399a4e6481e284c174" is_pre_election: true
I20260808 07:04:20.924220 19754 raft_consensus.cc:2454] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3363222887243e6b280bae428200dc1 in term 0.
I20260808 07:04:20.924345 20240 raft_consensus.cc:2781] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:20.924420 20240 raft_consensus.cc:482] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:20.924458 20240 raft_consensus.cc:3037] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.925135 20240 raft_consensus.cc:504] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.925235 20240 leader_election.cc:288] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [CANDIDATE]: Term 1 election: Requested vote from peers 3c98ca8862984f71a150782aeeefea78 (127.24.181.66:36851), 6b6bf43ea3b94e399a4e6481e284c174 (127.24.181.65:39033)
I20260808 07:04:20.925390 19934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa375d6aa05f4c8890879b26ab14cba6" candidate_uuid: "c3363222887243e6b280bae428200dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c98ca8862984f71a150782aeeefea78"
I20260808 07:04:20.925467 19754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa375d6aa05f4c8890879b26ab14cba6" candidate_uuid: "c3363222887243e6b280bae428200dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6bf43ea3b94e399a4e6481e284c174"
I20260808 07:04:20.925459 19934 raft_consensus.cc:3037] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.925544 19754 raft_consensus.cc:3037] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:20.926223 19754 raft_consensus.cc:2454] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3363222887243e6b280bae428200dc1 in term 1.
I20260808 07:04:20.926234 19934 raft_consensus.cc:2454] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3363222887243e6b280bae428200dc1 in term 1.
I20260808 07:04:20.926471 20034 leader_election.cc:302] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c98ca8862984f71a150782aeeefea78, c3363222887243e6b280bae428200dc1; no voters:
I20260808 07:04:20.926576 20240 raft_consensus.cc:2781] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:20.926733 20240 raft_consensus.cc:686] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 1 LEADER]: Becoming Leader. State: Replica: c3363222887243e6b280bae428200dc1, State: Running, Role: LEADER
I20260808 07:04:20.926849 20240 consensus_queue.cc:237] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } }
I20260808 07:04:20.927614 19553 catalog_manager.cc:5495] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3363222887243e6b280bae428200dc1 (127.24.181.67). New cstate: current_term: 1 leader_uuid: "c3363222887243e6b280bae428200dc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3363222887243e6b280bae428200dc1" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 40323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 } health_report { overall_health: UNKNOWN } } }
I20260808 07:04:20.938683 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.936288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.24.181.67:40323)
0808 07:04:20.936358 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:20.936360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:20.936369 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:20.936433 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:20.936433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:20.936456 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:20.936526 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:20.936653 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.937107 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.937916 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:20.938105 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:20.938200 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:20.938358 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:20.938444 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:20.938550 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:20.938581 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:04:20.938589 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:04:20.940368 19754 raft_consensus.cc:1261] T fa375d6aa05f4c8890879b26ab14cba6 P 6b6bf43ea3b94e399a4e6481e284c174 [term 1 FOLLOWER]: Refusing update from remote peer c3363222887243e6b280bae428200dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:20.940392 19934 raft_consensus.cc:1261] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 1 FOLLOWER]: Refusing update from remote peer c3363222887243e6b280bae428200dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:20.940706 20240 consensus_queue.cc:1035] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c98ca8862984f71a150782aeeefea78" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 36851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:20.940828 20244 consensus_queue.cc:1035] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b6bf43ea3b94e399a4e6481e284c174" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:20.943763 20250 mvcc.cc:205] Tried to move back new op lower bound from 7316163219209338880 to 7316163219157168128. Current Snapshot: MvccSnapshot[applied={T|T < 7316163219209338880}]
I20260808 07:04:20.944594 20251 mvcc.cc:205] Tried to move back new op lower bound from 7316163219209338880 to 7316163219157168128. Current Snapshot: MvccSnapshot[applied={T|T < 7316163219209338880}]
I20260808 07:04:20.945438 19552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:20.945530 19552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:20.946259 19552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb6fbc57bc3540d6b3f202ad283dc574: Sending DeleteTablet for 3 replicas of tablet fa375d6aa05f4c8890879b26ab14cba6
I20260808 07:04:20.946627 20104 tablet_service.cc:1514] Processing DeleteTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='slave'} at 127.0.0.1:34492
I20260808 07:04:20.946702 19728 tablet_service.cc:1514] Processing DeleteTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='slave'} at 127.0.0.1:59792
I20260808 07:04:20.946852 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 19637
I20260808 07:04:20.949569 19905 tablet_service.cc:1514] Processing DeleteTablet for tablet fa375d6aa05f4c8890879b26ab14cba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:20 UTC) from {username='slave'} at 127.0.0.1:54914
I20260808 07:04:20.949841 20269 tablet_replica.cc:330] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: stopping tablet replica
I20260808 07:04:20.949976 20269 raft_consensus.cc:2229] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:20.950039 20265 tablet_replica.cc:330] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: stopping tablet replica
I20260808 07:04:20.950142 20265 raft_consensus.cc:2229] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:20.950167 20269 raft_consensus.cc:2258] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:20.950261 20265 raft_consensus.cc:2258] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:20.952076 20269 ts_tablet_manager.cc:1905] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:20.952131 20265 ts_tablet_manager.cc:1905] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:20.953886 20265 ts_tablet_manager.cc:1918] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:20.953967 20265 log.cc:1196] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/fa375d6aa05f4c8890879b26ab14cba6
I20260808 07:04:20.954242 20265 ts_tablet_manager.cc:1939] T fa375d6aa05f4c8890879b26ab14cba6 P c3363222887243e6b280bae428200dc1: Deleting consensus metadata
I20260808 07:04:20.954444 20269 ts_tablet_manager.cc:1918] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:20.954572 20269 log.cc:1196] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/fa375d6aa05f4c8890879b26ab14cba6
I20260808 07:04:20.954649 19528 catalog_manager.cc:4841] TS c3363222887243e6b280bae428200dc1 (127.24.181.67:40323): tablet fa375d6aa05f4c8890879b26ab14cba6 (table test-table [id=f2c1edee3d5a4edd8a5533ff8683ffef]) successfully deleted
I20260808 07:04:20.955034 20269 ts_tablet_manager.cc:1939] T fa375d6aa05f4c8890879b26ab14cba6 P 3c98ca8862984f71a150782aeeefea78: Deleting consensus metadata
I20260808 07:04:20.955590 19528 catalog_manager.cc:4841] TS 3c98ca8862984f71a150782aeeefea78 (127.24.181.66:36851): tablet fa375d6aa05f4c8890879b26ab14cba6 (table test-table [id=f2c1edee3d5a4edd8a5533ff8683ffef]) successfully deleted
W20260808 07:04:20.960857 19528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:39033 (error 108)
I20260808 07:04:20.961167 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 19830
I20260808 07:04:20.961330 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:20.961195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:39033 (local address 127.0.0.1:59808)
0808 07:04:20.961253 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:20.961292 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:04:20.966858 19528 catalog_manager.cc:4584] TS 6b6bf43ea3b94e399a4e6481e284c174 (127.24.181.65:39033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa375d6aa05f4c8890879b26ab14cba6: Network error: Client connection negotiation failed: client connection to 127.24.181.65:39033: connect: Connection refused (error 111)
I20260808 07:04:20.977859 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 20009
I20260808 07:04:20.990301 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 19508
2026-08-08T07:04:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (707 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:21 dist-test-slave-nfb5 krb5kdc[20275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:21 dist-test-slave-nfb5 krb5kdc[20275](info): set up 2 sockets
Aug 08 07:04:21 dist-test-slave-nfb5 krb5kdc[20275](info): commencing operation
krb5kdc: starting...
W20260808 07:04:22.218465 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:22.339893 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36541
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:22.420783 20296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:22.421001 20296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:22.421025 20296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:22.422120 20296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:22.422154 20296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:22.422169 20296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:22.422183 20296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:22.422194 20296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:22.423714 20296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:36541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:36541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:22.423920 20296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:22.424099 20296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:22.429003 20301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.429011 20302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.429040 20296 server_base.cc:1034] running on GCE node
W20260808 07:04:22.429041 20305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.429862 20296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:22.430133 20296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:22.431290 20296 hybrid_clock.cc:648] HybridClock initialized: now 1786172662431274 us; error 52 us; skew 500 ppm
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:22.433772 20296 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:22.434230 20296 webserver.cc:460] Webserver started at http://127.24.181.126:34745/ using document root <none> and password file <none>
I20260808 07:04:22.434455 20296 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:22.434521 20296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:22.434669 20296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:22.435789 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "706eb0722a4a1dffc25c4eb0ae828a48"
server_key_iv: "99e658378379e57fe9284397c06e958f"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.436177 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "706eb0722a4a1dffc25c4eb0ae828a48"
server_key_iv: "99e658378379e57fe9284397c06e958f"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.437700 20296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:04:22.438449 20313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.438642 20296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:22.438752 20296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "706eb0722a4a1dffc25c4eb0ae828a48"
server_key_iv: "99e658378379e57fe9284397c06e958f"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.438843 20296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:22.458388 20296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:22.460398 20296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:22.460566 20296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:22.464751 20296 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:36541
I20260808 07:04:22.464797 20391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:36541 every 8 connection(s)
I20260808 07:04:22.465170 20296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:22.466023 20392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:22.468350 20392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: Bootstrap starting.
I20260808 07:04:22.468886 20392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:22.469121 20392 log.cc:824] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:22.469774 20392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: No bootstrap required, opened a new log
I20260808 07:04:22.471453 20392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } }
I20260808 07:04:22.471591 20392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:22.471613 20392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfe596b6bb54a8d94e0a3b0011b55b1, State: Initialized, Role: FOLLOWER
I20260808 07:04:22.471750 20392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } }
I20260808 07:04:22.471832 20392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:22.471855 20392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:22.471891 20392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:22.472389 20392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } }
I20260808 07:04:22.472477 20392 leader_election.cc:302] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bfe596b6bb54a8d94e0a3b0011b55b1; no voters:
I20260808 07:04:22.472615 20392 leader_election.cc:288] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:22.472678 20396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:22.472856 20392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:22.472873 20396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [term 1 LEADER]: Becoming Leader. State: Replica: 1bfe596b6bb54a8d94e0a3b0011b55b1, State: Running, Role: LEADER
I20260808 07:04:22.472996 20396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } }
I20260808 07:04:22.473407 20398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bfe596b6bb54a8d94e0a3b0011b55b1. Latest consensus state: current_term: 1 leader_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } } }
I20260808 07:04:22.473498 20398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:22.473649 20397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfe596b6bb54a8d94e0a3b0011b55b1" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 36541 } } }
I20260808 07:04:22.473776 20397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:22.473719 20407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:22.474426 20407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:22.474678 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 20296
I20260808 07:04:22.474805 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:22.475023 25301 external_mini_cluster.cc:1428] Setting key 5a449a58006037d5e876649a84a8a062
I20260808 07:04:22.476063 20407 catalog_manager.cc:1347] Generated new cluster ID: 1c32f28b3c384409a2a1770b327dc8ca
I20260808 07:04:22.476119 20407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:22.485184 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.475536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.24.181.126:36541)
0808 07:04:22.475964 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:22.475974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:22.476009 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:04:22.476141 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:22.476150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:22.476219 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:22.476365 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:22.477022 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.477964 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.478768 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.479284 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.479401 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.481803 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:22.481808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:22.481808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:22.481821 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:22.483590 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.483970 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.483974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.483975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.484079 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.484298 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.484299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.484299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.484536 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:22.484709 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:22.484807 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:04:22.484957 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":243,"threads_started":1}
I20260808 07:04:22.487689 20407 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:22.488402 20407 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:22.494000 20407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: Generated new TSK 0
I20260808 07:04:22.494166 20407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
Entry for principal kudu/127.24.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab.
I20260808 07:04:22.525451 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:22.609630 20429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:22.609871 20429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:22.609910 20429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:22.611109 20429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:22.611166 20429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:22.611209 20429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:22.612747 20429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.65.keytab
--principal=kudu/127.24.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:22.613014 20429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:22.613257 20429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:22.618296 20438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.618287 20435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.618469 20429 server_base.cc:1034] running on GCE node
W20260808 07:04:22.618284 20436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.618724 20429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:22.619009 20429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:22.620164 20429 hybrid_clock.cc:648] HybridClock initialized: now 1786172662620156 us; error 38 us; skew 500 ppm
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:22.622641 20429 init.cc:377] Logged in from keytab as kudu/127.24.181.65@KRBTEST.COM (short username kudu)
I20260808 07:04:22.623128 20429 webserver.cc:460] Webserver started at http://127.24.181.65:41239/ using document root <none> and password file <none>
I20260808 07:04:22.623385 20429 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:22.623461 20429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:22.623606 20429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:22.624643 20429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "39089ab332464d1b9bdc9bff0592c99b"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "bbae8408bc9bd5c224d6130741ccb6a6"
server_key_iv: "3c65fb90e1990b16cba87637da6a116a"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.624994 20429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "39089ab332464d1b9bdc9bff0592c99b"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "bbae8408bc9bd5c224d6130741ccb6a6"
server_key_iv: "3c65fb90e1990b16cba87637da6a116a"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.626276 20429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:22.627125 20452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.627292 20429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:22.627404 20429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "39089ab332464d1b9bdc9bff0592c99b"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "bbae8408bc9bd5c224d6130741ccb6a6"
server_key_iv: "3c65fb90e1990b16cba87637da6a116a"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.627483 20429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:22.645120 20429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:22.646955 20429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:22.647114 20429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:22.647357 20429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:22.647670 20429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:22.647722 20429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.647763 20429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:22.647804 20429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.653522 20429 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:38695
I20260808 07:04:22.653604 20624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:38695 every 8 connection(s)
I20260808 07:04:22.653904 20429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.65@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:22.661082 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 20429
I20260808 07:04:22.661216 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:22.661429 25301 external_mini_cluster.cc:1428] Setting key 9184ae2296b1ffe80efc392d6be69c8c
I20260808 07:04:22.663136 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.654578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:51687 (local address 127.24.181.126:36541)
0808 07:04:22.654653 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:22.654655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:22.654839 (+ 184us) server_negotiation.cc:407] Connection header received
0808 07:04:22.655349 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:22.655350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:22.655363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:22.655406 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:22.656284 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.656781 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.657728 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.657929 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.657985 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.660004 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:22.660006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:22.660006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:22.660013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:22.661640 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.662102 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.662103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.662103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.662172 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.662469 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.662470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.662470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.662571 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:22.662623 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:22.662997 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 07:04:22.663071 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:04:22.663424 20629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.654548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36541 (local address 127.24.181.65:51687)
0808 07:04:22.654727 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:22.654762 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:22.655257 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:22.655456 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:22.655459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:22.656020 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:22.656211 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.656215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.656857 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.656858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.657627 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.657628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.658002 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.658003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.658095 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.658552 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:22.658560 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:22.659919 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:22.661741 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.661749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.661751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.662030 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.662250 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.662251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.662251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.662400 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.662629 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:22.662636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:22.662798 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:04:22.662958 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 07:04:22.663146 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260808 07:04:22.663959 20627 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36541
I20260808 07:04:22.664072 20627 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:22.664263 20627 heartbeater.cc:511] Master 127.24.181.126:36541 requested a full tablet report, sending...
I20260808 07:04:22.664793 20338 ts_manager.cc:194] Registered new tserver with Master: 39089ab332464d1b9bdc9bff0592c99b (127.24.181.65:38695)
I20260808 07:04:22.665396 20338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.65@KRBTEST.COM'} at 127.24.181.65:51687
WARNING: no policy specified for kudu/127.24.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
Entry for principal kudu/127.24.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab.
I20260808 07:04:22.689131 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:22.771909 20635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:22.772178 20635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:22.772219 20635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:22.773783 20635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:22.773849 20635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:22.773896 20635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:22.775703 20635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.66.keytab
--principal=kudu/127.24.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:22.776002 20635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:22.776234 20635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:22.781363 20635 server_base.cc:1034] running on GCE node
W20260808 07:04:22.781304 20642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.781306 20645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.781354 20641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.781749 20635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:22.781948 20635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:22.783079 20635 hybrid_clock.cc:648] HybridClock initialized: now 1786172662783070 us; error 28 us; skew 500 ppm
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:22.785504 20635 init.cc:377] Logged in from keytab as kudu/127.24.181.66@KRBTEST.COM (short username kudu)
I20260808 07:04:22.785966 20635 webserver.cc:460] Webserver started at http://127.24.181.66:38733/ using document root <none> and password file <none>
I20260808 07:04:22.786196 20635 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:22.786269 20635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:22.786415 20635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:22.787568 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "4ad4918543d04ee6ae1aa6e45a601028"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "05bbee1b9b4acb80d5722f0f9b3eabf9"
server_key_iv: "9e788dad90118b59cefdec13e0ae6f99"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.787927 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "4ad4918543d04ee6ae1aa6e45a601028"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "05bbee1b9b4acb80d5722f0f9b3eabf9"
server_key_iv: "9e788dad90118b59cefdec13e0ae6f99"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.789295 20635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:22.790014 20654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.790179 20635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:22.790283 20635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "4ad4918543d04ee6ae1aa6e45a601028"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "05bbee1b9b4acb80d5722f0f9b3eabf9"
server_key_iv: "9e788dad90118b59cefdec13e0ae6f99"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.790354 20635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:22.814692 20635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:22.816624 20635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:22.816788 20635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:22.817034 20635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:22.817366 20635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:22.817426 20635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.817469 20635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:22.817500 20635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.823719 20635 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:34839
I20260808 07:04:22.823770 20828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:34839 every 8 connection(s)
I20260808 07:04:22.824146 20635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.66@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:22.833559 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.824879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:60845 (local address 127.24.181.126:36541)
0808 07:04:22.824927 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:22.824929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:22.825140 (+ 211us) server_negotiation.cc:407] Connection header received
0808 07:04:22.825746 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:22.825747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:22.825760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:22.825824 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:22.826817 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.827398 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.828393 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.828606 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.828691 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.830829 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:22.830831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:22.830831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:22.830839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:22.832182 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.832573 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.832575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.832575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.832637 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.832888 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.832889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.832890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.833011 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:22.833083 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:22.833369 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:04:22.833465 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:22.833717 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.824847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36541 (local address 127.24.181.66:60845)
0808 07:04:22.825003 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:22.825045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:22.825631 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:22.825895 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:22.825899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:22.826504 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:22.826705 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.826711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.827503 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.827504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.828305 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.828307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.828713 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.828715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.828835 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.829297 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:22.829305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:22.830738 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:22.832266 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.832274 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.832277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.832498 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.832707 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.832708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.832708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.832821 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.833087 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:22.833095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:22.833235 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:04:22.833333 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:04:22.833488 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 07:04:22.834187 20831 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36541
I20260808 07:04:22.834290 20831 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:22.834296 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 20635
I20260808 07:04:22.834443 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:22.834488 20831 heartbeater.cc:511] Master 127.24.181.126:36541 requested a full tablet report, sending...
I20260808 07:04:22.834682 25301 external_mini_cluster.cc:1428] Setting key 2f91c431b160e1aaff580525b11481d3
I20260808 07:04:22.834903 20338 ts_manager.cc:194] Registered new tserver with Master: 4ad4918543d04ee6ae1aa6e45a601028 (127.24.181.66:34839)
I20260808 07:04:22.835515 20338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.66@KRBTEST.COM'} at 127.24.181.66:60845
WARNING: no policy specified for kudu/127.24.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
Entry for principal kudu/127.24.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab.
I20260808 07:04:22.861351 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:22.944398 20839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:22.944622 20839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:22.944662 20839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:22.945801 20839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:22.945861 20839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:22.945909 20839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:22.947499 20839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.67.keytab
--principal=kudu/127.24.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:22.947754 20839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:22.947970 20839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:22.952984 20848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.953032 20846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:22.952987 20845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:22.953181 20839 server_base.cc:1034] running on GCE node
I20260808 07:04:22.953377 20839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:22.953591 20839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:22.954751 20839 hybrid_clock.cc:648] HybridClock initialized: now 1786172662954738 us; error 40 us; skew 500 ppm
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:22.957283 20839 init.cc:377] Logged in from keytab as kudu/127.24.181.67@KRBTEST.COM (short username kudu)
I20260808 07:04:22.957749 20839 webserver.cc:460] Webserver started at http://127.24.181.67:43957/ using document root <none> and password file <none>
I20260808 07:04:22.957999 20839 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:22.958065 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:22.958220 20839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:22.959339 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "fa86af6e2fb2411b939ef63fb7c72446"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "935dae5281b7fc93d9116636a1c5bc05"
server_key_iv: "91b31e794ee65730e13fd173aa7bd25d"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.959699 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "fa86af6e2fb2411b939ef63fb7c72446"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "935dae5281b7fc93d9116636a1c5bc05"
server_key_iv: "91b31e794ee65730e13fd173aa7bd25d"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.961054 20839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:04:22.961794 20858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.961967 20839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.962055 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "fa86af6e2fb2411b939ef63fb7c72446"
format_stamp: "Formatted at 2026-08-08 07:04:22 on dist-test-slave-nfb5"
server_key: "935dae5281b7fc93d9116636a1c5bc05"
server_key_iv: "91b31e794ee65730e13fd173aa7bd25d"
server_key_version: "encryptionkey@0"
I20260808 07:04:22.962129 20839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:22.977643 20839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:22.979611 20839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:22.979771 20839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:22.979991 20839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:22.980289 20839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:22.980340 20839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.980381 20839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:22.980410 20839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:22.986531 20839 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:46819
I20260808 07:04:22.986577 21049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:46819 every 8 connection(s)
I20260808 07:04:22.986900 20839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:04:22 dist-test-slave-nfb5 krb5kdc[20275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172662, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.67@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:22.995981 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.987610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:52591 (local address 127.24.181.126:36541)
0808 07:04:22.987674 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:22.987675 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:22.987912 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:04:22.988474 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:22.988475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:22.988490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:22.988541 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:22.989472 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.989982 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.991036 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.991245 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.991329 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.993401 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:22.993402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:22.993403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:22.993411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:22.994757 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.995212 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.995212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.995213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.995265 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:22.995476 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:22.995477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:22.995477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:22.995579 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:22.995630 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:22.995854 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:04:22.995917 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:04:22.996148 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:22.987593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:36541 (local address 127.24.181.67:52591)
0808 07:04:22.987777 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:22.987824 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:22.988362 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:22.988607 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:22.988617 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:22.989182 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:22.989371 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.989376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.990085 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.990091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.990880 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:22.990883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:22.991349 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:22.991350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:22.991468 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:22.991975 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:22.991985 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:22.993292 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:22.994874 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.994882 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.994884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.995145 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.995324 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:22.995325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:22.995325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:22.995424 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:22.995630 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:22.995635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:22.995746 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:04:22.995827 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:04:22.995956 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260808 07:04:22.996454 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 20839
I20260808 07:04:22.996579 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:22.996722 21050 heartbeater.cc:347] Connected to a master server at 127.24.181.126:36541
I20260808 07:04:22.996763 25301 external_mini_cluster.cc:1428] Setting key b9778478ab9dd6b9f33b4c1c8bef962f
I20260808 07:04:22.996831 21050 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:22.997020 21050 heartbeater.cc:511] Master 127.24.181.126:36541 requested a full tablet report, sending...
I20260808 07:04:22.997485 20338 ts_manager.cc:194] Registered new tserver with Master: fa86af6e2fb2411b939ef63fb7c72446 (127.24.181.67:46819)
I20260808 07:04:22.997895 20338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.181.67@KRBTEST.COM'} at 127.24.181.67:52591
I20260808 07:04:23.008419 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:23.015632 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.009714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.24.181.126:36541)
0808 07:04:23.009786 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.009789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.009902 (+ 113us) server_negotiation.cc:407] Connection header received
0808 07:04:23.009979 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.009980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.009991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.010041 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.010596 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.011121 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.011825 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.012020 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.012092 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.013253 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.013254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.013255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:23.013262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.014427 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:23.014756 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:23.014757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:23.014757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:23.014835 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:23.015067 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:23.015068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:23.015069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:23.015223 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.015341 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.015443 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.015530 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:04:23.017001 20338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:23.018510 20338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:23.025261 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.020870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:34839 (local address 127.0.0.1:33158)
0808 07:04:23.021214 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.021224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.021278 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.021693 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.021697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.021701 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.021768 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.021771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.023145 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.023146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.024278 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.024280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.024941 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.024942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.025023 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025036 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.025104 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.025161 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":33,"thread_start_us":83,"threads_started":1}
I20260808 07:04:23.025333 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.021004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.24.181.65:38695)
0808 07:04:23.021566 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.021571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.021583 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:23.021624 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.021626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.021643 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.021893 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.021973 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.022647 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.024206 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.024954 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.025062 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025116 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.025193 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.025246 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":105,"threads_started":1}
I20260808 07:04:23.025497 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.021206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.24.181.67:46819)
0808 07:04:23.021386 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.021392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.021430 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:04:23.021490 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.021492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.021508 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.021597 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.021860 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.023041 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.024236 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.024954 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.025030 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.025358 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.025405 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260808 07:04:23.025616 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.021020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.24.181.66:34839)
0808 07:04:23.021437 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.021441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.021451 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:23.021490 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.021492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.021508 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.021607 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.021877 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.023021 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.024354 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.024853 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.024940 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025002 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.025485 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.025531 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":89,"threads_started":1}
I20260808 07:04:23.025280 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.020870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:38695 (local address 127.0.0.1:58232)
0808 07:04:23.021050 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.021190 (+ 140us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.021278 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.021745 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.021746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.021748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.021812 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.021814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.022760 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.022761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.024115 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.024118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.025069 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.025070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.025129 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025131 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.025182 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.025215 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260808 07:04:23.025578 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.021085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:46819 (local address 127.0.0.1:39672)
0808 07:04:23.021378 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.021385 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.021430 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.021693 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.021697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.021699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.021768 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.021771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.023159 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.023160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.024117 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.024119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.025125 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.025126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.025193 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.025198 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.025248 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.025287 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":27,"thread_start_us":78,"threads_started":1}
I20260808 07:04:23.026503 20730 tablet_service.cc:1467] Processing CreateTablet for tablet d137b01e087444c1aa7f8924727b91d5 (DEFAULT_TABLE table=test-table [id=b650637d56894a60966e53b1af1fc344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.026500 20523 tablet_service.cc:1467] Processing CreateTablet for tablet d137b01e087444c1aa7f8924727b91d5 (DEFAULT_TABLE table=test-table [id=b650637d56894a60966e53b1af1fc344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.026844 20730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137b01e087444c1aa7f8924727b91d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.026844 20523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137b01e087444c1aa7f8924727b91d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.029392 21074 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Bootstrap starting.
I20260808 07:04:23.029626 21075 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Bootstrap starting.
I20260808 07:04:23.029978 20940 tablet_service.cc:1467] Processing CreateTablet for tablet d137b01e087444c1aa7f8924727b91d5 (DEFAULT_TABLE table=test-table [id=b650637d56894a60966e53b1af1fc344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.030206 21074 tablet_bootstrap.cc:654] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.030264 21075 tablet_bootstrap.cc:654] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.030306 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d137b01e087444c1aa7f8924727b91d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.030867 21075 log.cc:824] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.030865 21074 log.cc:824] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.031577 21075 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: No bootstrap required, opened a new log
I20260808 07:04:23.031625 21074 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: No bootstrap required, opened a new log
I20260808 07:04:23.031661 21075 ts_tablet_manager.cc:1397] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:04:23.031684 21074 ts_tablet_manager.cc:1397] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:23.032995 21075 raft_consensus.cc:348] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.033164 21075 raft_consensus.cc:374] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.033211 21075 raft_consensus.cc:729] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39089ab332464d1b9bdc9bff0592c99b, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.033340 21075 consensus_queue.cc:260] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.033634 21075 ts_tablet_manager.cc:1428] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:04:23.033656 20627 heartbeater.cc:503] Master 127.24.181.126:36541 was elected leader, sending a full tablet report...
I20260808 07:04:23.033733 21074 raft_consensus.cc:348] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.033911 21074 raft_consensus.cc:374] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.033939 21081 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Bootstrap starting.
I20260808 07:04:23.033969 21074 raft_consensus.cc:729] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad4918543d04ee6ae1aa6e45a601028, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.034080 21074 consensus_queue.cc:260] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.034379 21074 ts_tablet_manager.cc:1428] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:23.034444 20831 heartbeater.cc:503] Master 127.24.181.126:36541 was elected leader, sending a full tablet report...
I20260808 07:04:23.034616 21081 tablet_bootstrap.cc:654] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.035214 21081 log.cc:824] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.035921 21081 tablet_bootstrap.cc:492] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: No bootstrap required, opened a new log
I20260808 07:04:23.035975 21081 ts_tablet_manager.cc:1397] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:23.037256 21081 raft_consensus.cc:348] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.037384 21081 raft_consensus.cc:374] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.037444 21081 raft_consensus.cc:729] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa86af6e2fb2411b939ef63fb7c72446, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.037577 21081 consensus_queue.cc:260] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.037883 21081 ts_tablet_manager.cc:1428] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:23.037901 21050 heartbeater.cc:503] Master 127.24.181.126:36541 was elected leader, sending a full tablet report...
W20260808 07:04:23.074700 20832 tablet.cc:2369] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:23.154731 20628 tablet.cc:2369] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:23.202632 21080 raft_consensus.cc:482] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:23.202787 21080 raft_consensus.cc:504] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.203150 21080 leader_election.cc:288] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ad4918543d04ee6ae1aa6e45a601028 (127.24.181.66:34839), fa86af6e2fb2411b939ef63fb7c72446 (127.24.181.67:46819)
I20260808 07:04:23.207165 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.203476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:41525 (local address 127.24.181.67:46819)
0808 07:04:23.203520 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.203522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.203586 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:04:23.203648 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.203649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.203659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.203739 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.203964 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.204720 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206352 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.206742 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206799 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.206807 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.207041 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.207095 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:04:23.207177 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.203358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:46819 (local address 127.24.181.65:41525)
0808 07:04:23.203499 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.203512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.203585 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.203759 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.203760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.203762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.203885 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.203887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.204816 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.204817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.206252 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.206256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.206833 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.206963 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.206968 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.207022 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.207075 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":150,"thread_start_us":60,"threads_started":1}
I20260808 07:04:23.207228 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.203443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:51263 (local address 127.24.181.66:34839)
0808 07:04:23.203496 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.203498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.203587 (+ 89us) server_negotiation.cc:407] Connection header received
0808 07:04:23.203648 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.203649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.203659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.203739 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.203964 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.204725 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206386 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.206822 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206897 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.206908 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.207103 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.207149 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:04:23.207392 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.203358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:34839 (local address 127.24.181.65:51263)
0808 07:04:23.203501 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.203512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.203585 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.203759 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.203760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.203764 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.203893 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.203895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.204816 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.204817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.206252 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.206256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.206899 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.206900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.207019 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.207025 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.207085 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.207129 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":68,"thread_start_us":71,"threads_started":1}
I20260808 07:04:23.207582 20976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137b01e087444c1aa7f8924727b91d5" candidate_uuid: "39089ab332464d1b9bdc9bff0592c99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa86af6e2fb2411b939ef63fb7c72446" is_pre_election: true
I20260808 07:04:23.207773 20976 raft_consensus.cc:2454] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39089ab332464d1b9bdc9bff0592c99b in term 0.
I20260808 07:04:23.207747 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137b01e087444c1aa7f8924727b91d5" candidate_uuid: "39089ab332464d1b9bdc9bff0592c99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad4918543d04ee6ae1aa6e45a601028" is_pre_election: true
I20260808 07:04:23.207878 20761 raft_consensus.cc:2454] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39089ab332464d1b9bdc9bff0592c99b in term 0.
I20260808 07:04:23.208029 20458 leader_election.cc:302] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39089ab332464d1b9bdc9bff0592c99b, fa86af6e2fb2411b939ef63fb7c72446; no voters:
I20260808 07:04:23.208187 21080 raft_consensus.cc:2781] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:23.208278 21080 raft_consensus.cc:482] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:23.208315 21080 raft_consensus.cc:3037] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.209151 21080 raft_consensus.cc:504] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.209303 21080 leader_election.cc:288] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [CANDIDATE]: Term 1 election: Requested vote from peers 4ad4918543d04ee6ae1aa6e45a601028 (127.24.181.66:34839), fa86af6e2fb2411b939ef63fb7c72446 (127.24.181.67:46819)
I20260808 07:04:23.209471 20976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137b01e087444c1aa7f8924727b91d5" candidate_uuid: "39089ab332464d1b9bdc9bff0592c99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa86af6e2fb2411b939ef63fb7c72446"
I20260808 07:04:23.209499 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d137b01e087444c1aa7f8924727b91d5" candidate_uuid: "39089ab332464d1b9bdc9bff0592c99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad4918543d04ee6ae1aa6e45a601028"
I20260808 07:04:23.209548 20976 raft_consensus.cc:3037] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.209611 20761 raft_consensus.cc:3037] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.210392 20976 raft_consensus.cc:2454] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39089ab332464d1b9bdc9bff0592c99b in term 1.
I20260808 07:04:23.210402 20761 raft_consensus.cc:2454] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39089ab332464d1b9bdc9bff0592c99b in term 1.
I20260808 07:04:23.210597 20460 leader_election.cc:302] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39089ab332464d1b9bdc9bff0592c99b, 4ad4918543d04ee6ae1aa6e45a601028; no voters:
I20260808 07:04:23.210711 21080 raft_consensus.cc:2781] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:23.210875 21080 raft_consensus.cc:686] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [term 1 LEADER]: Becoming Leader. State: Replica: 39089ab332464d1b9bdc9bff0592c99b, State: Running, Role: LEADER
I20260808 07:04:23.210997 21080 consensus_queue.cc:237] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } }
I20260808 07:04:23.211803 20338 catalog_manager.cc:5495] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 39089ab332464d1b9bdc9bff0592c99b (127.24.181.65). New cstate: current_term: 1 leader_uuid: "39089ab332464d1b9bdc9bff0592c99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39089ab332464d1b9bdc9bff0592c99b" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 } health_report { overall_health: UNKNOWN } } }
W20260808 07:04:23.237540 21051 tablet.cc:2369] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:23.255622 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.253168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.24.181.65:38695)
0808 07:04:23.253231 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.253233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.253243 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:23.253298 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.253298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.253328 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.253406 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:23.253557 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.254035 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.254803 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.255018 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.255101 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.255284 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:23.255393 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:23.255488 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.255507 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.255528 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:04:23.257556 20976 raft_consensus.cc:1261] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 1 FOLLOWER]: Refusing update from remote peer 39089ab332464d1b9bdc9bff0592c99b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:23.257557 20761 raft_consensus.cc:1261] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 1 FOLLOWER]: Refusing update from remote peer 39089ab332464d1b9bdc9bff0592c99b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:23.257918 21080 consensus_queue.cc:1035] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ad4918543d04ee6ae1aa6e45a601028" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:23.258085 21092 consensus_queue.cc:1035] T d137b01e087444c1aa7f8924727b91d5 P 39089ab332464d1b9bdc9bff0592c99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa86af6e2fb2411b939ef63fb7c72446" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 46819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:23.260708 21096 mvcc.cc:205] Tried to move back new op lower bound from 7316163228699832320 to 7316163228512907264. Current Snapshot: MvccSnapshot[applied={T|T < 7316163228699832320}]
I20260808 07:04:23.263307 20338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:23.263417 20338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:23.264289 20338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bfe596b6bb54a8d94e0a3b0011b55b1: Sending DeleteTablet for 3 replicas of tablet d137b01e087444c1aa7f8924727b91d5
I20260808 07:04:23.264652 20730 tablet_service.cc:1514] Processing DeleteTablet for tablet d137b01e087444c1aa7f8924727b91d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:33158
I20260808 07:04:23.264724 20523 tablet_service.cc:1514] Processing DeleteTablet for tablet d137b01e087444c1aa7f8924727b91d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:58232
I20260808 07:04:23.264982 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet d137b01e087444c1aa7f8924727b91d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='kudu', principal='kudu/127.24.181.126@KRBTEST.COM'} at 127.0.0.1:39672
I20260808 07:04:23.265089 21110 tablet_replica.cc:330] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: stopping tablet replica
I20260808 07:04:23.265266 21113 tablet_replica.cc:330] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: stopping tablet replica
I20260808 07:04:23.265360 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 20429
I20260808 07:04:23.265393 21113 raft_consensus.cc:2229] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:23.265498 21113 raft_consensus.cc:2258] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:23.266012 21113 ts_tablet_manager.cc:1905] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:23.268237 21113 ts_tablet_manager.cc:1918] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:23.268316 21113 log.cc:1196] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/d137b01e087444c1aa7f8924727b91d5
I20260808 07:04:23.268589 21113 ts_tablet_manager.cc:1939] T d137b01e087444c1aa7f8924727b91d5 P fa86af6e2fb2411b939ef63fb7c72446: Deleting consensus metadata
I20260808 07:04:23.268996 20317 catalog_manager.cc:4841] TS fa86af6e2fb2411b939ef63fb7c72446 (127.24.181.67:46819): tablet d137b01e087444c1aa7f8924727b91d5 (table test-table [id=b650637d56894a60966e53b1af1fc344]) successfully deleted
I20260808 07:04:23.269078 21110 raft_consensus.cc:2229] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:23.269217 21110 raft_consensus.cc:2258] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:23.269711 21110 ts_tablet_manager.cc:1905] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:23.271850 21110 ts_tablet_manager.cc:1918] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:23.271941 21110 log.cc:1196] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/d137b01e087444c1aa7f8924727b91d5
I20260808 07:04:23.272217 21110 ts_tablet_manager.cc:1939] T d137b01e087444c1aa7f8924727b91d5 P 4ad4918543d04ee6ae1aa6e45a601028: Deleting consensus metadata
I20260808 07:04:23.272634 20319 catalog_manager.cc:4841] TS 4ad4918543d04ee6ae1aa6e45a601028 (127.24.181.66:34839): tablet d137b01e087444c1aa7f8924727b91d5 (table test-table [id=b650637d56894a60966e53b1af1fc344]) successfully deleted
W20260808 07:04:23.276422 20317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:38695 (error 108)
I20260808 07:04:23.276664 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 20635
I20260808 07:04:23.279508 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.279331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:38695 (local address 127.0.0.1:58254)
0808 07:04:23.279401 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.279469 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 07:04:23.279613 20317 catalog_manager.cc:4584] TS 39089ab332464d1b9bdc9bff0592c99b (127.24.181.65:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d137b01e087444c1aa7f8924727b91d5: Network error: Client connection negotiation failed: client connection to 127.24.181.65:38695: connect: Connection refused (error 111)
I20260808 07:04:23.288776 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 20839
I20260808 07:04:23.297778 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 20296
2026-08-08T07:04:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2309 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T07:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:23Z Disabled control of system clock
I20260808 07:04:23.329967 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34951
--webserver_interface=127.24.181.126
--webserver_port=0
--builtin_ntp_servers=127.24.181.84:35919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:23.411154 21119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:23.411356 21119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:23.411376 21119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:23.412427 21119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:23.412460 21119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:23.412473 21119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:23.412485 21119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:23.412497 21119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:23.413762 21119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:23.413998 21119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:23.414240 21119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:23.419337 21130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:23.419375 21127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.419754 21119 server_base.cc:1034] running on GCE node
W20260808 07:04:23.419379 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.420053 21119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:23.420328 21119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:23.421483 21119 hybrid_clock.cc:648] HybridClock initialized: now 1786172663421458 us; error 36 us; skew 500 ppm
I20260808 07:04:23.422861 21119 webserver.cc:460] Webserver started at http://127.24.181.126:36741/ using document root <none> and password file <none>
I20260808 07:04:23.423198 21119 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:23.423256 21119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:23.423445 21119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:23.424634 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "e8654c52946e4e319c234fb025bfcb73"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "260affb5c6684d80ec756bfd59bf6f72"
server_key_iv: "d4790b6c3a027013ace96d239d619823"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.425033 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "e8654c52946e4e319c234fb025bfcb73"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "260affb5c6684d80ec756bfd59bf6f72"
server_key_iv: "d4790b6c3a027013ace96d239d619823"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.426427 21119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:23.427268 21140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.427448 21119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:23.427536 21119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "e8654c52946e4e319c234fb025bfcb73"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "260affb5c6684d80ec756bfd59bf6f72"
server_key_iv: "d4790b6c3a027013ace96d239d619823"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.427595 21119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:23.443185 21119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:23.443486 21119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:23.443595 21119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:23.447779 21119 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:34951
I20260808 07:04:23.447839 21217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:34951 every 8 connection(s)
I20260808 07:04:23.448206 21119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:23.449347 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.451762 21218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: Bootstrap starting.
I20260808 07:04:23.452316 21218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.452581 21218 log.cc:824] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.453281 21218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: No bootstrap required, opened a new log
I20260808 07:04:23.454535 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 21119
I20260808 07:04:23.454661 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:23.454861 25301 external_mini_cluster.cc:1428] Setting key 0c20d59fec4267aac65f41d773954558
I20260808 07:04:23.455031 21218 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } }
I20260808 07:04:23.455235 21218 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.455281 21218 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8654c52946e4e319c234fb025bfcb73, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.455430 21218 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } }
I20260808 07:04:23.455524 21218 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:23.455569 21218 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:23.455610 21218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.456133 21218 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } }
I20260808 07:04:23.456262 21218 leader_election.cc:302] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8654c52946e4e319c234fb025bfcb73; no voters:
I20260808 07:04:23.456440 21218 leader_election.cc:288] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:23.456487 21224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:23.456624 21224 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [term 1 LEADER]: Becoming Leader. State: Replica: e8654c52946e4e319c234fb025bfcb73, State: Running, Role: LEADER
I20260808 07:04:23.456723 21218 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:23.456710 21224 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } }
I20260808 07:04:23.457063 21224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8654c52946e4e319c234fb025bfcb73. Latest consensus state: current_term: 1 leader_uuid: "e8654c52946e4e319c234fb025bfcb73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } } }
I20260808 07:04:23.457154 21224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:23.457075 21228 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8654c52946e4e319c234fb025bfcb73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8654c52946e4e319c234fb025bfcb73" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34951 } } }
I20260808 07:04:23.457314 21228 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:23.457383 21232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:23.457952 21232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:23.460105 21232 catalog_manager.cc:1347] Generated new cluster ID: 9220edfdfece41d2a0dd2db989dfa92f
I20260808 07:04:23.460163 21232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:23.461241 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.455443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.24.181.126:34951)
0808 07:04:23.455687 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.455697 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.455728 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:04:23.455845 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.455850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.455909 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.456042 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.456267 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.457954 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.459084 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.459565 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.459748 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.460511 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.460515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.460515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:23.460720 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.460760 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:23.460773 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.460842 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.460946 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.461018 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260808 07:04:23.473922 21232 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:23.474630 21232 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:23.479142 21232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: Generated new TSK 0
I20260808 07:04:23.479324 21232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:23.486987 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.65:0
--local_ip_for_outbound_sockets=127.24.181.65
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--builtin_ntp_servers=127.24.181.84:35919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:23.570801 21250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:23.571067 21250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:23.571111 21250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:23.572248 21250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:23.572307 21250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:23.572350 21250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.65
I20260808 07:04:23.573693 21250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.181.65
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.65
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:23.573946 21250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:23.574247 21250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:23.580107 21255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:23.580063 21257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.580389 21250 server_base.cc:1034] running on GCE node
W20260808 07:04:23.580070 21261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.580683 21250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:23.580952 21250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:23.582116 21250 hybrid_clock.cc:648] HybridClock initialized: now 1786172663582096 us; error 37 us; skew 500 ppm
I20260808 07:04:23.583405 21250 webserver.cc:460] Webserver started at http://127.24.181.65:42627/ using document root <none> and password file <none>
I20260808 07:04:23.583658 21250 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:23.583762 21250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:23.583937 21250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:23.585282 21250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data/instance:
uuid: "8ddc1adccaa04713a9d904e95870de93"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "a0fb59a56867c0baf14344f0bfe00c3d"
server_key_iv: "532856a05556c14feede2bc1fe2e6c53"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.585646 21250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance:
uuid: "8ddc1adccaa04713a9d904e95870de93"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "a0fb59a56867c0baf14344f0bfe00c3d"
server_key_iv: "532856a05556c14feede2bc1fe2e6c53"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.587004 21250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:23.587782 21268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.587970 21250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:23.588058 21250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal
uuid: "8ddc1adccaa04713a9d904e95870de93"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "a0fb59a56867c0baf14344f0bfe00c3d"
server_key_iv: "532856a05556c14feede2bc1fe2e6c53"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.588142 21250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:23.595398 21250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:23.595661 21250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:23.595810 21250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:23.596040 21250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:23.596342 21250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:23.596395 21250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.596434 21250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:23.596463 21250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.602488 21250 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.65:38471
I20260808 07:04:23.602545 21444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.65:38471 every 8 connection(s)
I20260808 07:04:23.602880 21250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:23.607801 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.603650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:46737 (local address 127.24.181.126:34951)
0808 07:04:23.603697 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.603699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.603937 (+ 238us) server_negotiation.cc:407] Connection header received
0808 07:04:23.604385 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.604386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.604397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.604454 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.605067 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.605557 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.606494 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.606685 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.606748 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.607549 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.607551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.607551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:23.607557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.607572 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:23.607576 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.607621 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.607698 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.607734 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:23.607966 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.603615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34951 (local address 127.24.181.65:46737)
0808 07:04:23.603803 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.603851 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.604305 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.604502 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.604505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.604514 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:23.604996 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.605000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.605635 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.605636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.606411 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.606413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.606764 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.606765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.606880 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.607421 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:23.607422 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:23.607448 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:23.607470 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:23.607628 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:23.607632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:23.607635 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.607686 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.607764 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260808 07:04:23.608494 21445 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34951
I20260808 07:04:23.608615 21445 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:23.608814 21445 heartbeater.cc:511] Master 127.24.181.126:34951 requested a full tablet report, sending...
I20260808 07:04:23.609373 21167 ts_manager.cc:194] Registered new tserver with Master: 8ddc1adccaa04713a9d904e95870de93 (127.24.181.65:38471)
I20260808 07:04:23.610033 21167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.65:46737
I20260808 07:04:23.612600 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 21250
I20260808 07:04:23.612723 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:23.612927 25301 external_mini_cluster.cc:1428] Setting key 8ad1738f424dea90db696eda95ca2617
I20260808 07:04:23.613833 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.66:0
--local_ip_for_outbound_sockets=127.24.181.66
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--builtin_ntp_servers=127.24.181.84:35919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:23.696424 21448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:23.696617 21448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:23.696638 21448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:23.697696 21448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:23.697728 21448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:23.697753 21448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.66
I20260808 07:04:23.699108 21448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.181.66
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.66
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:23.699290 21448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:23.699540 21448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:23.704527 21455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:23.704535 21454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.704552 21448 server_base.cc:1034] running on GCE node
W20260808 07:04:23.704535 21457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.705003 21448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:23.705196 21448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:23.706336 21448 hybrid_clock.cc:648] HybridClock initialized: now 1786172663706320 us; error 33 us; skew 500 ppm
I20260808 07:04:23.707530 21448 webserver.cc:460] Webserver started at http://127.24.181.66:40463/ using document root <none> and password file <none>
I20260808 07:04:23.707751 21448 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:23.707791 21448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:23.707883 21448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:23.708941 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data/instance:
uuid: "0c4afc1dd0dd4369ae77f89784053238"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "51e1fb242d06198056b297a0de859599"
server_key_iv: "97badfcf5795121dc4e07cdcb606e60d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.709232 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance:
uuid: "0c4afc1dd0dd4369ae77f89784053238"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "51e1fb242d06198056b297a0de859599"
server_key_iv: "97badfcf5795121dc4e07cdcb606e60d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.710500 21448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:23.711246 21467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.711400 21448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:23.711486 21448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal
uuid: "0c4afc1dd0dd4369ae77f89784053238"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "51e1fb242d06198056b297a0de859599"
server_key_iv: "97badfcf5795121dc4e07cdcb606e60d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.711540 21448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:23.734665 21448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:23.735011 21448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:23.735126 21448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:23.735318 21448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:23.735666 21448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:23.735709 21448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.735764 21448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:23.735790 21448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.741617 21448 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.66:38473
I20260808 07:04:23.741696 21640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.66:38473 every 8 connection(s)
I20260808 07:04:23.742018 21448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:23.747344 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.742768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.66:42519 (local address 127.24.181.126:34951)
0808 07:04:23.742833 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.742836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.743067 (+ 231us) server_negotiation.cc:407] Connection header received
0808 07:04:23.743625 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.743627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.743640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.743700 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.744397 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.744903 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.745899 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.746100 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.746176 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.747048 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.747050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.747050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:23.747055 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.747071 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:23.747075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.747138 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.747232 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.747263 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:23.747561 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.742740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34951 (local address 127.24.181.66:42519)
0808 07:04:23.742945 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.742978 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.743504 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.743770 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.743774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.743784 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:23.744289 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.744294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.745005 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.745006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.745800 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.745802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.746186 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.746187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.746331 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.746892 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:23.746893 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:23.746938 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:23.746960 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:23.747140 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:23.747146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:23.747150 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.747234 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.747343 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260808 07:04:23.748044 21642 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34951
I20260808 07:04:23.748149 21642 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:23.748350 21642 heartbeater.cc:511] Master 127.24.181.126:34951 requested a full tablet report, sending...
I20260808 07:04:23.748728 21167 ts_manager.cc:194] Registered new tserver with Master: 0c4afc1dd0dd4369ae77f89784053238 (127.24.181.66:38473)
I20260808 07:04:23.749089 21167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.66:42519
I20260808 07:04:23.749392 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 21448
I20260808 07:04:23.749513 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:23.749763 25301 external_mini_cluster.cc:1428] Setting key 7bcbd10e072c33aa7c98bd8af4afbfb3
I20260808 07:04:23.750577 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.181.67:0
--local_ip_for_outbound_sockets=127.24.181.67
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--builtin_ntp_servers=127.24.181.84:35919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:04:23.833839 21645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:23.834056 21645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:23.834079 21645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:23.835264 21645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:23.835306 21645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:23.835331 21645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.181.67
I20260808 07:04:23.836709 21645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.181.67
--webserver_port=0
--tserver_master_addrs=127.24.181.126:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.181.67
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:23.836918 21645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:23.837203 21645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:23.842190 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:23.842221 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:23.842244 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:23.842453 21645 server_base.cc:1034] running on GCE node
I20260808 07:04:23.842730 21645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:23.843011 21645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:23.844162 21645 hybrid_clock.cc:648] HybridClock initialized: now 1786172663844136 us; error 45 us; skew 500 ppm
I20260808 07:04:23.845413 21645 webserver.cc:460] Webserver started at http://127.24.181.67:44273/ using document root <none> and password file <none>
I20260808 07:04:23.845657 21645 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:23.845703 21645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:23.845842 21645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:23.847008 21645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data/instance:
uuid: "b230513ee938461497f2ad2bae4832e0"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "abaae0d1ac8c641af76b1201816067f3"
server_key_iv: "fffeed50044ef0bd5fc5d218abad963d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.847383 21645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance:
uuid: "b230513ee938461497f2ad2bae4832e0"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "abaae0d1ac8c641af76b1201816067f3"
server_key_iv: "fffeed50044ef0bd5fc5d218abad963d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.848723 21645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:04:23.849398 21665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.849565 21645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:23.849659 21645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal
uuid: "b230513ee938461497f2ad2bae4832e0"
format_stamp: "Formatted at 2026-08-08 07:04:23 on dist-test-slave-nfb5"
server_key: "abaae0d1ac8c641af76b1201816067f3"
server_key_iv: "fffeed50044ef0bd5fc5d218abad963d"
server_key_version: "encryptionkey@0"
I20260808 07:04:23.849728 21645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:23.863667 21645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:23.863958 21645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:23.864097 21645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:23.864326 21645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:23.864673 21645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:23.864724 21645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.864758 21645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:23.864786 21645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:23.870726 21645 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.67:38599
I20260808 07:04:23.870780 21835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.67:38599 every 8 connection(s)
I20260808 07:04:23.871119 21645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:23.875993 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 21645
I20260808 07:04:23.876122 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:23.876374 25301 external_mini_cluster.cc:1428] Setting key 8180cafb86a64e30dd41382bab4a4dd9
I20260808 07:04:23.876376 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.871828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.67:34877 (local address 127.24.181.126:34951)
0808 07:04:23.871877 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.871879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.872123 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:04:23.872674 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.872676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.872688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.872748 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.873431 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.873937 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.874906 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.875125 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.875200 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.876052 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.876053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.876054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:23.876059 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.876076 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:23.876085 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.876150 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.876287 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.876315 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:04:23.876646 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.871800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34951 (local address 127.24.181.67:34877)
0808 07:04:23.872000 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.872033 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.872566 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.872820 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.872824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.872833 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:23.873312 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.873316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.874030 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.874031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.874812 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.874814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.875210 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.875211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.875348 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.875900 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:04:23.875900 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:23.875927 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:04:23.875949 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:23.876181 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:23.876189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:23.876193 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.876291 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.876405 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260808 07:04:23.877108 21836 heartbeater.cc:347] Connected to a master server at 127.24.181.126:34951
I20260808 07:04:23.877205 21836 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:23.877408 21836 heartbeater.cc:511] Master 127.24.181.126:34951 requested a full tablet report, sending...
I20260808 07:04:23.877741 21167 ts_manager.cc:194] Registered new tserver with Master: b230513ee938461497f2ad2bae4832e0 (127.24.181.67:38599)
I20260808 07:04:23.878075 21167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.181.67:34877
I20260808 07:04:23.888164 25301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:23.892642 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.889605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.24.181.126:34951)
0808 07:04:23.889682 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.889684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.889787 (+ 103us) server_negotiation.cc:407] Connection header received
0808 07:04:23.889870 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.889871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.889884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.889934 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:23.890104 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.890602 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.891465 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.891663 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.891749 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.892378 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:23.892379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:23.892380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:23.892384 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:23.892399 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:23.892403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:23.892463 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.892531 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.892557 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:04:23.894287 21167 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:23.895921 21167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:23.902020 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.897948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:38599 (local address 127.0.0.1:60090)
0808 07:04:23.898280 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.898292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.898405 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.898632 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.898635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.898638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.898711 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.898713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.899721 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.899722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.900850 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.900853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.901745 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.901746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.901802 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.901811 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.901892 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.901945 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":113,"threads_started":1}
I20260808 07:04:23.902033 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.898036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.24.181.67:38599)
0808 07:04:23.898231 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.898237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.898369 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:04:23.898509 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.898512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.898530 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.898617 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.898867 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.899638 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.900954 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.901599 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.901636 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.901695 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.901911 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.901949 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260808 07:04:23.902474 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.898151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:38471 (local address 127.0.0.1:45844)
0808 07:04:23.898297 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.898306 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.898347 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.898816 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.898817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.898819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.898863 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.898865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.900220 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.900221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.901245 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.901248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.902269 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.902270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.902326 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.902330 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.902361 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.902401 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260808 07:04:23.902740 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.897937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:38473 (local address 127.0.0.1:52060)
0808 07:04:23.898138 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.898181 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.898250 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.898724 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.898725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.898727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.898779 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.898781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.900163 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.900164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.901224 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.901226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.902415 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.902416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.902472 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.902474 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.902641 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.902680 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":57,"threads_started":1}
I20260808 07:04:23.902814 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.898032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.24.181.66:38473)
0808 07:04:23.898550 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.898554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.898562 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:23.898616 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.898618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.898633 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.898718 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.898869 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.900038 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.901307 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.902026 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.902294 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.902389 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.902676 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.902721 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":84,"threads_started":1}
I20260808 07:04:23.902767 21737 tablet_service.cc:1467] Processing CreateTablet for tablet 969a269cab264b679052bd7d04d1e1c5 (DEFAULT_TABLE table=test-table [id=defb12b843e84d69af975d5e33be3a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.902968 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.898298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.24.181.65:38471)
0808 07:04:23.898514 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.898519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.898529 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:23.898568 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.898569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.898585 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.898967 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.899027 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.900151 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.901354 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.902058 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.902655 (+ 597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.902762 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.902789 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.902843 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":119,"threads_started":1}
I20260808 07:04:23.903218 21737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969a269cab264b679052bd7d04d1e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.903539 21339 tablet_service.cc:1467] Processing CreateTablet for tablet 969a269cab264b679052bd7d04d1e1c5 (DEFAULT_TABLE table=test-table [id=defb12b843e84d69af975d5e33be3a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.903802 21339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969a269cab264b679052bd7d04d1e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.906050 21860 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Bootstrap starting.
I20260808 07:04:23.906412 21862 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: Bootstrap starting.
I20260808 07:04:23.906440 21536 tablet_service.cc:1467] Processing CreateTablet for tablet 969a269cab264b679052bd7d04d1e1c5 (DEFAULT_TABLE table=test-table [id=defb12b843e84d69af975d5e33be3a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:23.906773 21536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969a269cab264b679052bd7d04d1e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:23.906828 21860 tablet_bootstrap.cc:654] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.907076 21862 tablet_bootstrap.cc:654] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.907192 21860 log.cc:824] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.907397 21862 log.cc:824] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.907917 21860 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: No bootstrap required, opened a new log
I20260808 07:04:23.908000 21860 ts_tablet_manager.cc:1397] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:23.908154 21862 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: No bootstrap required, opened a new log
I20260808 07:04:23.908231 21862 ts_tablet_manager.cc:1397] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:04:23.910332 21866 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Bootstrap starting.
I20260808 07:04:23.910277 21862 raft_consensus.cc:348] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.910456 21862 raft_consensus.cc:374] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.910503 21862 raft_consensus.cc:729] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddc1adccaa04713a9d904e95870de93, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.910643 21862 consensus_queue.cc:260] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.910688 21860 raft_consensus.cc:348] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.910854 21860 raft_consensus.cc:374] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.910897 21860 raft_consensus.cc:729] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b230513ee938461497f2ad2bae4832e0, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.910972 21862 ts_tablet_manager.cc:1428] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:04:23.911037 21860 consensus_queue.cc:260] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.911098 21866 tablet_bootstrap.cc:654] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:23.911109 21445 heartbeater.cc:503] Master 127.24.181.126:34951 was elected leader, sending a full tablet report...
I20260808 07:04:23.911299 21860 ts_tablet_manager.cc:1428] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:04:23.911334 21836 heartbeater.cc:503] Master 127.24.181.126:34951 was elected leader, sending a full tablet report...
I20260808 07:04:23.911460 21866 log.cc:824] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:23.912392 21866 tablet_bootstrap.cc:492] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: No bootstrap required, opened a new log
I20260808 07:04:23.912448 21866 ts_tablet_manager.cc:1397] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:23.914491 21866 raft_consensus.cc:348] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.914608 21866 raft_consensus.cc:374] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:23.914628 21866 raft_consensus.cc:729] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4afc1dd0dd4369ae77f89784053238, State: Initialized, Role: FOLLOWER
I20260808 07:04:23.914698 21866 consensus_queue.cc:260] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.914884 21866 ts_tablet_manager.cc:1428] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:04:23.914978 21642 heartbeater.cc:503] Master 127.24.181.126:34951 was elected leader, sending a full tablet report...
I20260808 07:04:23.977777 21867 raft_consensus.cc:482] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:23.977893 21867 raft_consensus.cc:504] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.978214 21867 leader_election.cc:288] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c4afc1dd0dd4369ae77f89784053238 (127.24.181.66:38473), b230513ee938461497f2ad2bae4832e0 (127.24.181.67:38599)
I20260808 07:04:23.981092 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.978407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.67:38599 (local address 127.24.181.65:32987)
0808 07:04:23.978478 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.978491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.978538 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.978698 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.978699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.978701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.978780 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.978782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.979490 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.979491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.980258 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.980260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.980820 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.980821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.980911 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.980913 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.980978 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.981020 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:04:23.981092 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.978480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:32987 (local address 127.24.181.67:38599)
0808 07:04:23.978549 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.978551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.978564 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:23.978600 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.978601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.978614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.978690 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.978864 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.979394 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.980354 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.980726 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.980819 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.980828 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.980984 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.981020 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:04:23.981333 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.978520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.181.65:36621 (local address 127.24.181.66:38473)
0808 07:04:23.978568 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.978570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.978793 (+ 223us) server_negotiation.cc:407] Connection header received
0808 07:04:23.978874 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.978875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.978888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.978992 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:23.979155 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.979686 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.980609 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.980989 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.981072 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.981081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.981225 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.981283 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:04:23.981369 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.978407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.66:38473 (local address 127.24.181.65:36621)
0808 07:04:23.978710 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:23.978720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:23.978794 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:23.978999 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:23.979000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:23.979002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:23.979076 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.979078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.979775 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.979776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.980518 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:23.980520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.981079 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.981080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:23.981153 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.981157 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:23.981215 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:04:23.981266 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":105,"thread_start_us":143,"threads_started":1}
I20260808 07:04:23.981560 21770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969a269cab264b679052bd7d04d1e1c5" candidate_uuid: "8ddc1adccaa04713a9d904e95870de93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b230513ee938461497f2ad2bae4832e0" is_pre_election: true
I20260808 07:04:23.981712 21770 raft_consensus.cc:2454] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ddc1adccaa04713a9d904e95870de93 in term 0.
I20260808 07:04:23.981791 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969a269cab264b679052bd7d04d1e1c5" candidate_uuid: "8ddc1adccaa04713a9d904e95870de93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4afc1dd0dd4369ae77f89784053238" is_pre_election: true
I20260808 07:04:23.981948 21569 raft_consensus.cc:2454] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ddc1adccaa04713a9d904e95870de93 in term 0.
I20260808 07:04:23.981972 21274 leader_election.cc:302] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ddc1adccaa04713a9d904e95870de93, b230513ee938461497f2ad2bae4832e0; no voters:
I20260808 07:04:23.982093 21867 raft_consensus.cc:2781] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:23.982187 21867 raft_consensus.cc:482] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:23.982218 21867 raft_consensus.cc:3037] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.982988 21867 raft_consensus.cc:504] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.983131 21867 leader_election.cc:288] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [CANDIDATE]: Term 1 election: Requested vote from peers 0c4afc1dd0dd4369ae77f89784053238 (127.24.181.66:38473), b230513ee938461497f2ad2bae4832e0 (127.24.181.67:38599)
I20260808 07:04:23.983266 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969a269cab264b679052bd7d04d1e1c5" candidate_uuid: "8ddc1adccaa04713a9d904e95870de93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4afc1dd0dd4369ae77f89784053238"
I20260808 07:04:23.983325 21770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969a269cab264b679052bd7d04d1e1c5" candidate_uuid: "8ddc1adccaa04713a9d904e95870de93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b230513ee938461497f2ad2bae4832e0"
I20260808 07:04:23.983366 21569 raft_consensus.cc:3037] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.983395 21770 raft_consensus.cc:3037] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:23.984333 21770 raft_consensus.cc:2454] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ddc1adccaa04713a9d904e95870de93 in term 1.
I20260808 07:04:23.984411 21569 raft_consensus.cc:2454] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ddc1adccaa04713a9d904e95870de93 in term 1.
I20260808 07:04:23.984511 21274 leader_election.cc:302] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ddc1adccaa04713a9d904e95870de93, b230513ee938461497f2ad2bae4832e0; no voters:
I20260808 07:04:23.984638 21867 raft_consensus.cc:2781] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:23.984828 21867 raft_consensus.cc:686] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 1 LEADER]: Becoming Leader. State: Replica: 8ddc1adccaa04713a9d904e95870de93, State: Running, Role: LEADER
I20260808 07:04:23.984949 21867 consensus_queue.cc:237] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } }
I20260808 07:04:23.985805 21167 catalog_manager.cc:5495] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ddc1adccaa04713a9d904e95870de93 (127.24.181.65). New cstate: current_term: 1 leader_uuid: "8ddc1adccaa04713a9d904e95870de93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ddc1adccaa04713a9d904e95870de93" member_type: VOTER last_known_addr { host: "127.24.181.65" port: 38471 } health_report { overall_health: HEALTHY } } }
W20260808 07:04:23.992519 21643 tablet.cc:2369] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:23.992945 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:23.990551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.24.181.65:38471)
0808 07:04:23.990594 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:23.990596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:23.990606 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:23.990682 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:23.990683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:23.990727 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:23.990801 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:23.990942 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.991378 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.992205 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:23.992398 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:23.992479 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:23.992643 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:23.992758 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:23.992843 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:23.992864 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 07:04:23.992883 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:04:23.994544 21770 raft_consensus.cc:1261] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 1 FOLLOWER]: Refusing update from remote peer 8ddc1adccaa04713a9d904e95870de93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:23.994550 21569 raft_consensus.cc:1261] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 1 FOLLOWER]: Refusing update from remote peer 8ddc1adccaa04713a9d904e95870de93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:23.994863 21867 consensus_queue.cc:1035] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4afc1dd0dd4369ae77f89784053238" member_type: VOTER last_known_addr { host: "127.24.181.66" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:23.995015 21876 consensus_queue.cc:1035] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b230513ee938461497f2ad2bae4832e0" member_type: VOTER last_known_addr { host: "127.24.181.67" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:23.997197 21879 mvcc.cc:205] Tried to move back new op lower bound from 7316163231719178240 to 7316163231682985984. Current Snapshot: MvccSnapshot[applied={T|T < 7316163231719178240}]
I20260808 07:04:23.998111 21881 mvcc.cc:205] Tried to move back new op lower bound from 7316163231719178240 to 7316163231682985984. Current Snapshot: MvccSnapshot[applied={T|T < 7316163231719178240}]
I20260808 07:04:23.999855 21167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:23.999949 21167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:24.000741 21167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8654c52946e4e319c234fb025bfcb73: Sending DeleteTablet for 3 replicas of tablet 969a269cab264b679052bd7d04d1e1c5
I20260808 07:04:24.001050 21339 tablet_service.cc:1514] Processing DeleteTablet for tablet 969a269cab264b679052bd7d04d1e1c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='slave'} at 127.0.0.1:45844
I20260808 07:04:24.001101 21737 tablet_service.cc:1514] Processing DeleteTablet for tablet 969a269cab264b679052bd7d04d1e1c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='slave'} at 127.0.0.1:60090
I20260808 07:04:24.001138 21536 tablet_service.cc:1514] Processing DeleteTablet for tablet 969a269cab264b679052bd7d04d1e1c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:23 UTC) from {username='slave'} at 127.0.0.1:52060
I20260808 07:04:24.001340 21892 tablet_replica.cc:330] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: stopping tablet replica
I20260808 07:04:24.001412 21890 tablet_replica.cc:330] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93: stopping tablet replica
I20260808 07:04:24.001519 21892 raft_consensus.cc:2229] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:24.001518 21890 raft_consensus.cc:2229] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:24.001579 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 21250
I20260808 07:04:24.001618 21890 raft_consensus.cc:2258] T 969a269cab264b679052bd7d04d1e1c5 P 8ddc1adccaa04713a9d904e95870de93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:24.001736 21892 raft_consensus.cc:2258] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:24.001922 21893 tablet_replica.cc:330] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: stopping tablet replica
I20260808 07:04:24.002028 21893 raft_consensus.cc:2229] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:24.002148 21893 raft_consensus.cc:2258] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:24.010372 21893 ts_tablet_manager.cc:1905] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:24.011255 21892 ts_tablet_manager.cc:1905] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:24.013579 21892 ts_tablet_manager.cc:1918] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:24.013737 21892 log.cc:1196] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-2/wal/wals/969a269cab264b679052bd7d04d1e1c5
I20260808 07:04:24.014168 21892 ts_tablet_manager.cc:1939] T 969a269cab264b679052bd7d04d1e1c5 P b230513ee938461497f2ad2bae4832e0: Deleting consensus metadata
I20260808 07:04:24.014703 21148 catalog_manager.cc:4841] TS b230513ee938461497f2ad2bae4832e0 (127.24.181.67:38599): tablet 969a269cab264b679052bd7d04d1e1c5 (table test-table [id=defb12b843e84d69af975d5e33be3a43]) successfully deleted
I20260808 07:04:24.015523 21893 ts_tablet_manager.cc:1918] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:24.015588 21893 log.cc:1196] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Deleting WAL directory at /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172407597382-25301-0/minicluster-data/ts-1/wal/wals/969a269cab264b679052bd7d04d1e1c5
I20260808 07:04:24.015820 21893 ts_tablet_manager.cc:1939] T 969a269cab264b679052bd7d04d1e1c5 P 0c4afc1dd0dd4369ae77f89784053238: Deleting consensus metadata
I20260808 07:04:24.016239 21147 catalog_manager.cc:4841] TS 0c4afc1dd0dd4369ae77f89784053238 (127.24.181.66:38473): tablet 969a269cab264b679052bd7d04d1e1c5 (table test-table [id=defb12b843e84d69af975d5e33be3a43]) successfully deleted
W20260808 07:04:24.018378 21142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.181.65:38471 (error 108)
I20260808 07:04:24.018654 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 21448
I20260808 07:04:24.018860 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:24.018724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.65:38471 (local address 127.0.0.1:45854)
0808 07:04:24.018779 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:24.018820 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.65:38471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:04:24.018991 21142 catalog_manager.cc:4584] TS 8ddc1adccaa04713a9d904e95870de93 (127.24.181.65:38471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 969a269cab264b679052bd7d04d1e1c5: Network error: Client connection negotiation failed: client connection to 127.24.181.65:38471: connect: Connection refused (error 111)
I20260808 07:04:24.031710 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 21645
I20260808 07:04:24.039139 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 21119
2026-08-08T07:04:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (739 ms)
[----------] 6 tests from EncryptionPolicyTest (9395 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:24 dist-test-slave-nfb5 krb5kdc[21900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:24 dist-test-slave-nfb5 krb5kdc[21900](info): set up 2 sockets
Aug 08 07:04:24 dist-test-slave-nfb5 krb5kdc[21900](info): commencing operation
krb5kdc: starting...
W20260808 07:04:25.262403 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:25.388254 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:25.474606 21916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:25.474848 21916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:25.474888 21916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:25.476073 21916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:25.476143 21916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:25.476167 21916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:25.476202 21916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:25.476231 21916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:25.477723 21916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:25.478003 21916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:25.478236 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:25.483201 21930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:25.483217 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:25.483271 21933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:25.483605 21916 server_base.cc:1034] running on GCE node
I20260808 07:04:25.483896 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:25.484196 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:25.485370 21916 hybrid_clock.cc:648] HybridClock initialized: now 1786172665485365 us; error 39 us; skew 500 ppm
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:25.490728 21916 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:25.491268 21916 webserver.cc:460] Webserver started at http://127.0.0.1:44027/ using document root <none> and password file <none>
I20260808 07:04:25.491534 21916 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:25.491608 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:25.491757 21916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:25.492836 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "d5aef142a6f44441863484289193eae2"
format_stamp: "Formatted at 2026-08-08 07:04:25 on dist-test-slave-nfb5"
server_key: "19a7fbbc03a362838e9bf215a9aba951"
server_key_iv: "9ac9de54fc6f2320ccff8a1e2353cc7b"
server_key_version: "encryptionkey@0"
I20260808 07:04:25.493213 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "d5aef142a6f44441863484289193eae2"
format_stamp: "Formatted at 2026-08-08 07:04:25 on dist-test-slave-nfb5"
server_key: "19a7fbbc03a362838e9bf215a9aba951"
server_key_iv: "9ac9de54fc6f2320ccff8a1e2353cc7b"
server_key_version: "encryptionkey@0"
I20260808 07:04:25.494699 21916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:25.495563 21941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:25.495765 21916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:25.495888 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "d5aef142a6f44441863484289193eae2"
format_stamp: "Formatted at 2026-08-08 07:04:25 on dist-test-slave-nfb5"
server_key: "19a7fbbc03a362838e9bf215a9aba951"
server_key_iv: "9ac9de54fc6f2320ccff8a1e2353cc7b"
server_key_version: "encryptionkey@0"
I20260808 07:04:25.495967 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:25.534842 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:25.536857 21916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:25.537038 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:25.541216 21916 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35197
I20260808 07:04:25.541327 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35197 every 8 connection(s)
I20260808 07:04:25.541654 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:25.542604 22018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:25.544257 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 21916
I20260808 07:04:25.544390 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:25.544595 25301 external_mini_cluster.cc:1428] Setting key 338dd196298948a9a4b1d83f8381837b
I20260808 07:04:25.545651 22018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2: Bootstrap starting.
I20260808 07:04:25.546360 22018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:25.546713 22018 log.cc:824] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:25.547536 22018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2: No bootstrap required, opened a new log
I20260808 07:04:25.549854 22018 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 07:04:25.550028 22018 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:25.550073 22018 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5aef142a6f44441863484289193eae2, State: Initialized, Role: FOLLOWER
I20260808 07:04:25.550195 22018 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 07:04:25.550300 22018 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:25.550344 22018 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:25.550397 22018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:25.551002 22018 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 07:04:25.551121 22018 leader_election.cc:302] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5aef142a6f44441863484289193eae2; no voters:
I20260808 07:04:25.551327 22018 leader_election.cc:288] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:25.551352 22028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:25.551471 22028 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [term 1 LEADER]: Becoming Leader. State: Replica: d5aef142a6f44441863484289193eae2, State: Running, Role: LEADER
I20260808 07:04:25.551555 22028 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 07:04:25.551612 22018 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:04:25.551884 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5aef142a6f44441863484289193eae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } } }
I20260808 07:04:25.551968 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:25.552153 22030 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5aef142a6f44441863484289193eae2. Latest consensus state: current_term: 1 leader_uuid: "d5aef142a6f44441863484289193eae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aef142a6f44441863484289193eae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } } }
I20260808 07:04:25.552232 22030 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:25.552217 22035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:25.552978 22035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:25.554636 22035 catalog_manager.cc:1347] Generated new cluster ID: 6700f75897c44dcbba5d69507040c4f1
I20260808 07:04:25.554688 22035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:25.556604 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:25.545181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38429 (local address 127.0.0.1:35197)
0808 07:04:25.545389 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:25.545397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:25.545425 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:04:25.545894 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:25.545901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:25.545948 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:25.546062 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:25.547007 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:25.548037 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:25.549112 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:25.549508 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:25.549586 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:25.552547 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:25.552551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:25.552552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:25.552565 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:25.554904 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:25.555363 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:25.555366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:25.555367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:25.555471 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:25.555733 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:25.555734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:25.555735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:25.556007 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:25.556147 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:25.556317 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:04:25.556412 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260808 07:04:25.557471 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:25.566176 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:25.558988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33963 (local address 127.0.0.1:35197)
0808 07:04:25.559031 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:25.559032 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:25.559139 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:04:25.559164 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:25.559165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:25.559176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:25.559259 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:25.559844 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:25.560586 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:25.561597 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:25.561938 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:25.562013 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:25.563302 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:25.563303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:25.563304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:25.563311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:25.564897 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:25.565311 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:25.565312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:25.565313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:25.565373 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:25.565605 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:25.565606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:25.565606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:25.565734 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:25.565885 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:25.566019 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:04:25.566088 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:25.567301 22035 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:25.567792 22035 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:25.571496 22035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5aef142a6f44441863484289193eae2: Generated new TSK 0
I20260808 07:04:25.571686 22035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:25.579005 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 21916
W20260808 07:04:25.586297 21896 connection.cc:535] client connection to 127.0.0.1:35197 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35197): Connection reset by peer (error 104)
2026-08-08T07:04:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[22062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[22062](info): set up 2 sockets
Aug 08 07:04:25 dist-test-slave-nfb5 krb5kdc[22062](info): commencing operation
krb5kdc: starting...
W20260808 07:04:26.797405 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:26 dist-test-slave-nfb5 krb5kdc[22062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:26.917990 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:26.999840 22089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:27.000043 22089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:27.000066 22089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:27.001144 22089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:27.001181 22089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:27.001195 22089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:27.001209 22089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:27.001220 22089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:27.002609 22089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:27.002799 22089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:27.002990 22089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:27.007687 22094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:27.007692 22096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:27.007721 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:27.008076 22089 server_base.cc:1034] running on GCE node
I20260808 07:04:27.008280 22089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:27.008518 22089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:27.009662 22089 hybrid_clock.cc:648] HybridClock initialized: now 1786172667009644 us; error 29 us; skew 500 ppm
Aug 08 07:04:27 dist-test-slave-nfb5 krb5kdc[22062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:27.012253 22089 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:27.012759 22089 webserver.cc:460] Webserver started at http://127.0.0.1:43291/ using document root <none> and password file <none>
I20260808 07:04:27.013018 22089 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:27.013092 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:27.013244 22089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:27.014302 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "a6a8a48da5af4d6289986b84e96e184f"
format_stamp: "Formatted at 2026-08-08 07:04:27 on dist-test-slave-nfb5"
server_key: "a237c6217bef90b43f1318b1df00dd0c"
server_key_iv: "c48daaab7b1910bfd8aa7f47786cad55"
server_key_version: "encryptionkey@0"
I20260808 07:04:27.014688 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "a6a8a48da5af4d6289986b84e96e184f"
format_stamp: "Formatted at 2026-08-08 07:04:27 on dist-test-slave-nfb5"
server_key: "a237c6217bef90b43f1318b1df00dd0c"
server_key_iv: "c48daaab7b1910bfd8aa7f47786cad55"
server_key_version: "encryptionkey@0"
I20260808 07:04:27.016129 22089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:27.016939 22111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:27.017138 22089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:27.017241 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "a6a8a48da5af4d6289986b84e96e184f"
format_stamp: "Formatted at 2026-08-08 07:04:27 on dist-test-slave-nfb5"
server_key: "a237c6217bef90b43f1318b1df00dd0c"
server_key_iv: "c48daaab7b1910bfd8aa7f47786cad55"
server_key_version: "encryptionkey@0"
I20260808 07:04:27.017334 22089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:27.027086 22089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:27.028892 22089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:27.029055 22089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:27.033423 22089 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41665
I20260808 07:04:27.033458 22175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41665 every 8 connection(s)
I20260808 07:04:27.033811 22089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:27.034719 22179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:27.037106 22179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: Bootstrap starting.
I20260808 07:04:27.037724 22179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:27.038041 22179 log.cc:824] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:27.038755 22179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: No bootstrap required, opened a new log
I20260808 07:04:27.040402 22179 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } }
I20260808 07:04:27.040541 22179 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:27.040565 22179 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a8a48da5af4d6289986b84e96e184f, State: Initialized, Role: FOLLOWER
I20260808 07:04:27.040753 22179 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } }
I20260808 07:04:27.040817 22179 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:27.040839 22179 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:27.040905 22179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:27.041451 22179 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } }
I20260808 07:04:27.041568 22179 leader_election.cc:302] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6a8a48da5af4d6289986b84e96e184f; no voters:
I20260808 07:04:27.041755 22179 leader_election.cc:288] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:27.041785 22183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:27.041990 22183 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [term 1 LEADER]: Becoming Leader. State: Replica: a6a8a48da5af4d6289986b84e96e184f, State: Running, Role: LEADER
I20260808 07:04:27.042012 22179 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:27.042083 22183 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } }
I20260808 07:04:27.042793 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22089
I20260808 07:04:27.042968 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:27.043206 25301 external_mini_cluster.cc:1428] Setting key 881dec0b51c5ba9e1539329bf52af726
I20260808 07:04:27.043632 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6a8a48da5af4d6289986b84e96e184f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } } }
I20260808 07:04:27.044034 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [sys.catalog]: This master's current role is: LEADER
W20260808 07:04:27.044092 22205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:27.044157 22205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:27.043833 22185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6a8a48da5af4d6289986b84e96e184f. Latest consensus state: current_term: 1 leader_uuid: "a6a8a48da5af4d6289986b84e96e184f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6a8a48da5af4d6289986b84e96e184f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41665 } } }
I20260808 07:04:27.044190 22185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:27.044324 22210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:27.044492 22210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:27.046068 22210 catalog_manager.cc:1347] Generated new cluster ID: af118bd6f32b445f9906590358851f45
I20260808 07:04:27.046125 22210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:27.049665 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:27.043740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47893 (local address 127.0.0.1:41665)
0808 07:04:27.043959 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:27.043969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:27.044265 (+ 296us) server_negotiation.cc:407] Connection header received
0808 07:04:27.044394 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:27.044401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:27.044460 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:27.044666 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:27.044888 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:27.046108 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:27.047342 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:27.047930 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:27.048045 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:27.048955 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:27.048959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:27.048959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:27.049159 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:27.049194 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:27.049208 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:27.049285 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:27.049423 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:04:27.049452 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260808 07:04:27.050671 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:27.055757 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:27.052265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44175 (local address 127.0.0.1:41665)
0808 07:04:27.052324 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:27.052326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:27.052390 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:04:27.052464 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:27.052465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:27.052474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:27.052531 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:27.052675 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:27.053435 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:27.054428 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:27.054733 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:27.054789 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:27.055501 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:27.055502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:27.055503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:27.055507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:27.055521 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:27.055526 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:27.055583 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:27.055675 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:04:27.055686 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:04:27.060109 22210 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:27.060838 22210 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:27.064502 22210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6a8a48da5af4d6289986b84e96e184f: Generated new TSK 0
I20260808 07:04:27.064693 22210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:27.069818 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22089
W20260808 07:04:27.079875 22059 connection.cc:535] client connection to 127.0.0.1:41665 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41665): Connection reset by peer (error 104)
2026-08-08T07:04:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:27 dist-test-slave-nfb5 krb5kdc[22223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:27 dist-test-slave-nfb5 krb5kdc[22223](info): set up 2 sockets
Aug 08 07:04:27 dist-test-slave-nfb5 krb5kdc[22223](info): commencing operation
krb5kdc: starting...
W20260808 07:04:28.291638 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:28 dist-test-slave-nfb5 krb5kdc[22223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:28.415377 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:28.496336 22241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:28.496588 22241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:28.496627 22241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:28.497766 22241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:28.497855 22241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:28.497884 22241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:28.497906 22241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:28.498044 22241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:28.499814 22241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:28.500202 22241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:28.500463 22241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:28.505457 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:28.505447 22256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:28.505448 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:28.505535 22241 server_base.cc:1034] running on GCE node
I20260808 07:04:28.505895 22241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:28.506173 22241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:28.507395 22241 hybrid_clock.cc:648] HybridClock initialized: now 1786172668507378 us; error 26 us; skew 500 ppm
Aug 08 07:04:28 dist-test-slave-nfb5 krb5kdc[22223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:28.509917 22241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:28.510349 22241 webserver.cc:460] Webserver started at http://127.0.0.1:37495/ using document root <none> and password file <none>
I20260808 07:04:28.510603 22241 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:28.510681 22241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:28.510834 22241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:28.511926 22241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "0404bb3b97624853852ac87ec327bcdd"
format_stamp: "Formatted at 2026-08-08 07:04:28 on dist-test-slave-nfb5"
server_key: "de9b1370e9dd62f5a5f220dbdeba33e1"
server_key_iv: "c258716a750aab6e1603d8eaed4d8934"
server_key_version: "encryptionkey@0"
I20260808 07:04:28.512298 22241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "0404bb3b97624853852ac87ec327bcdd"
format_stamp: "Formatted at 2026-08-08 07:04:28 on dist-test-slave-nfb5"
server_key: "de9b1370e9dd62f5a5f220dbdeba33e1"
server_key_iv: "c258716a750aab6e1603d8eaed4d8934"
server_key_version: "encryptionkey@0"
I20260808 07:04:28.513612 22241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:28.514317 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:28.514494 22241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:28.514604 22241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "0404bb3b97624853852ac87ec327bcdd"
format_stamp: "Formatted at 2026-08-08 07:04:28 on dist-test-slave-nfb5"
server_key: "de9b1370e9dd62f5a5f220dbdeba33e1"
server_key_iv: "c258716a750aab6e1603d8eaed4d8934"
server_key_version: "encryptionkey@0"
I20260808 07:04:28.514680 22241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:28.517935 22241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:28.519918 22241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:28.520064 22241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:28.524086 22241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44065
I20260808 07:04:28.524142 22331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44065 every 8 connection(s)
I20260808 07:04:28.524477 22241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:28.525466 22334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:28.527730 22334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd: Bootstrap starting.
I20260808 07:04:28.528303 22334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:28.528620 22334 log.cc:824] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:28.529316 22334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd: No bootstrap required, opened a new log
I20260808 07:04:28.530169 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22241
I20260808 07:04:28.530326 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:28.530531 25301 external_mini_cluster.cc:1428] Setting key f4b1395ac3f748df8fd80af1f49019cb
I20260808 07:04:28.531615 22334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260808 07:04:28.531785 22334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:28.531844 22334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0404bb3b97624853852ac87ec327bcdd, State: Initialized, Role: FOLLOWER
I20260808 07:04:28.531973 22334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260808 07:04:28.532068 22334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:28.532109 22334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:28.532150 22334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:28.532964 22334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260808 07:04:28.532994 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:28.531133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51337 (local address 127.0.0.1:44065)
0808 07:04:28.531469 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:28.531480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:28.531514 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:04:28.531629 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:28.531635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:28.531666 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:28.531818 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:28.532547 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:28.532551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:28.532551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:28.532555 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:28.532593 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:28.532606 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:28.532684 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:28.532746 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:04:28.532786 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":210,"threads_started":1}
I20260808 07:04:28.533087 22334 leader_election.cc:302] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0404bb3b97624853852ac87ec327bcdd; no voters:
I20260808 07:04:28.533246 22334 leader_election.cc:288] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:28.533319 22339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:28.533452 22339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [term 1 LEADER]: Becoming Leader. State: Replica: 0404bb3b97624853852ac87ec327bcdd, State: Running, Role: LEADER
I20260808 07:04:28.533532 22334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:28.533542 22339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260808 07:04:28.533946 22339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0404bb3b97624853852ac87ec327bcdd. Latest consensus state: current_term: 1 leader_uuid: "0404bb3b97624853852ac87ec327bcdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } } }
I20260808 07:04:28.534013 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:28.534044 22339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:28.533958 22340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0404bb3b97624853852ac87ec327bcdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0404bb3b97624853852ac87ec327bcdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } } }
I20260808 07:04:28.534171 22340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:28.534264 22348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:28.535377 22348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:28.536900 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:28.535640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42179 (local address 127.0.0.1:44065)
0808 07:04:28.535672 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:28.535674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:28.535933 (+ 259us) server_negotiation.cc:407] Connection header received
0808 07:04:28.536000 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:28.536001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:28.536006 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:28.536055 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:28.536689 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:28.536690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:28.536690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:28.536695 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:28.536708 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:28.536713 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:28.536759 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:28.536821 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:04:28.536836 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:04:28.539589 22348 catalog_manager.cc:1347] Generated new cluster ID: d07f1969578742719dba3da327c51a90
I20260808 07:04:28.539661 22348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:28.552222 22348 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:28.552407 22348 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:28.563135 22348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0404bb3b97624853852ac87ec327bcdd: Generated new TSK 0
I20260808 07:04:28.563357 22348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:28.571743 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22241
2026-08-08T07:04:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:28 dist-test-slave-nfb5 krb5kdc[22382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:28 dist-test-slave-nfb5 krb5kdc[22382](info): set up 2 sockets
Aug 08 07:04:28 dist-test-slave-nfb5 krb5kdc[22382](info): commencing operation
krb5kdc: starting...
W20260808 07:04:29.792806 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:29 dist-test-slave-nfb5 krb5kdc[22382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:29.917521 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:29.999788 22401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:30.000017 22401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:30.000041 22401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:30.001161 22401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:30.001196 22401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:30.001211 22401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:30.001224 22401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:30.001236 22401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:30.002631 22401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:30.002830 22401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:30.003026 22401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:30.007893 22409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:30.007894 22412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:30.007929 22408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:30.008074 22401 server_base.cc:1034] running on GCE node
I20260808 07:04:30.008296 22401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:30.008594 22401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:30.009793 22401 hybrid_clock.cc:648] HybridClock initialized: now 1786172670009777 us; error 34 us; skew 500 ppm
Aug 08 07:04:30 dist-test-slave-nfb5 krb5kdc[22382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:30.012488 22401 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:30.013005 22401 webserver.cc:460] Webserver started at http://127.0.0.1:41523/ using document root <none> and password file <none>
I20260808 07:04:30.013257 22401 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:30.013331 22401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:30.013480 22401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:30.014593 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c823a569ce6b4962bfdc28acf0e305a8"
format_stamp: "Formatted at 2026-08-08 07:04:30 on dist-test-slave-nfb5"
server_key: "58721f1383caad0886b2940894b800d1"
server_key_iv: "532898776713914eababa07e57d2f22f"
server_key_version: "encryptionkey@0"
I20260808 07:04:30.015050 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c823a569ce6b4962bfdc28acf0e305a8"
format_stamp: "Formatted at 2026-08-08 07:04:30 on dist-test-slave-nfb5"
server_key: "58721f1383caad0886b2940894b800d1"
server_key_iv: "532898776713914eababa07e57d2f22f"
server_key_version: "encryptionkey@0"
I20260808 07:04:30.016479 22401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:30.017432 22422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:30.017688 22401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:30.017802 22401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c823a569ce6b4962bfdc28acf0e305a8"
format_stamp: "Formatted at 2026-08-08 07:04:30 on dist-test-slave-nfb5"
server_key: "58721f1383caad0886b2940894b800d1"
server_key_iv: "532898776713914eababa07e57d2f22f"
server_key_version: "encryptionkey@0"
I20260808 07:04:30.017887 22401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:30.078474 22401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:30.080947 22401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:30.081128 22401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:30.085448 22401 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41209
I20260808 07:04:30.085495 22489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41209 every 8 connection(s)
I20260808 07:04:30.085886 22401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:30.086825 22490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:30.089238 22490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8: Bootstrap starting.
I20260808 07:04:30.089856 22490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:30.090147 22490 log.cc:824] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:30.090899 22490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8: No bootstrap required, opened a new log
I20260808 07:04:30.092710 22490 raft_consensus.cc:348] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260808 07:04:30.092847 22490 raft_consensus.cc:374] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:30.092869 22490 raft_consensus.cc:729] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c823a569ce6b4962bfdc28acf0e305a8, State: Initialized, Role: FOLLOWER
I20260808 07:04:30.093039 22490 consensus_queue.cc:260] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260808 07:04:30.093103 22490 raft_consensus.cc:388] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:30.093124 22490 raft_consensus.cc:482] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:30.093197 22490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:30.093627 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22401
I20260808 07:04:30.093775 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:30.093779 22490 raft_consensus.cc:504] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260808 07:04:30.093894 22490 leader_election.cc:302] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c823a569ce6b4962bfdc28acf0e305a8; no voters:
I20260808 07:04:30.094009 25301 external_mini_cluster.cc:1428] Setting key 72583539a9e08722ac98be22be922afb
I20260808 07:04:30.094085 22490 leader_election.cc:288] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:30.094125 22494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:30.094260 22494 raft_consensus.cc:686] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [term 1 LEADER]: Becoming Leader. State: Replica: c823a569ce6b4962bfdc28acf0e305a8, State: Running, Role: LEADER
I20260808 07:04:30.094362 22490 sys_catalog.cc:564] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:30.094409 22494 consensus_queue.cc:237] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260808 07:04:30.094805 22495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c823a569ce6b4962bfdc28acf0e305a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260808 07:04:30.094894 22495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:30.095157 22504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:30.095142 22498 sys_catalog.cc:455] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c823a569ce6b4962bfdc28acf0e305a8. Latest consensus state: current_term: 1 leader_uuid: "c823a569ce6b4962bfdc28acf0e305a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c823a569ce6b4962bfdc28acf0e305a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260808 07:04:30.095311 22498 sys_catalog.cc:458] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:30.096001 22504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:30.097535 22504 catalog_manager.cc:1347] Generated new cluster ID: dc4734990f1449b0af189e3f22741977
I20260808 07:04:30.097602 22504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:30 dist-test-slave-nfb5 krb5kdc[22382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:04:30.106706 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:30.095101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40953 (local address 127.0.0.1:41209)
0808 07:04:30.095280 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:30.095289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:30.095333 (+ 44us) server_negotiation.cc:407] Connection header received
0808 07:04:30.095452 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:30.095459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:30.095514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:30.096381 (+ 867us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:30.096488 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:30.097568 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:30.098672 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:30.099275 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:30.099373 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:30.102371 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:30.102375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:30.102376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:30.102388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:30.104894 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:30.105439 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:30.105443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:30.105443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:30.105563 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:30.105793 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:30.105793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:30.105794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:30.106072 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:30.106227 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:30.106374 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:04:30.106501 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260808 07:04:30.107923 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:30.119411 22504 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:30.120049 22504 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:30.122992 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:30.109348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43011 (local address 127.0.0.1:41209)
0808 07:04:30.109380 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:30.109381 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:30.115141 (+ 5760us) server_negotiation.cc:407] Connection header received
0808 07:04:30.115219 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:30.115219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:30.115231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:30.115295 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:30.116028 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:30.116774 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:30.117890 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:30.118210 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:30.118280 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:30.119860 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:30.119861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:30.119862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:30.119869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:30.121708 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:30.122159 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:30.122160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:30.122160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:30.122219 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:30.122436 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:30.122437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:30.122437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:30.122553 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:30.122651 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:30.122789 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:04:30.122883 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:04:30.129599 22504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c823a569ce6b4962bfdc28acf0e305a8: Generated new TSK 0
I20260808 07:04:30.129815 22504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:30.132400 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22401
2026-08-08T07:04:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:30 dist-test-slave-nfb5 krb5kdc[22533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:30 dist-test-slave-nfb5 krb5kdc[22533](info): set up 2 sockets
Aug 08 07:04:30 dist-test-slave-nfb5 krb5kdc[22533](info): commencing operation
krb5kdc: starting...
W20260808 07:04:31.380659 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:31 dist-test-slave-nfb5 krb5kdc[22533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:31.504258 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:31.585886 22549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:31.586100 22549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:31.586123 22549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:31.587312 22549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:31.587364 22549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:31.587378 22549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:31.587391 22549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:31.587450 22549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:31.588958 22549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:31.589202 22549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:31.589457 22549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:31.594519 22557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:31.594504 22555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:31.594492 22554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:31.594535 22549 server_base.cc:1034] running on GCE node
I20260808 07:04:31.595007 22549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:31.595275 22549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:31.596428 22549 hybrid_clock.cc:648] HybridClock initialized: now 1786172671596413 us; error 32 us; skew 500 ppm
Aug 08 07:04:31 dist-test-slave-nfb5 krb5kdc[22533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:31.599083 22549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:31.599488 22549 webserver.cc:460] Webserver started at http://127.0.0.1:37637/ using document root <none> and password file <none>
I20260808 07:04:31.599675 22549 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:31.599725 22549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:31.599821 22549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:31.600816 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "46598b70251c4a9bbfd3a2cc6d40d206"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-nfb5"
server_key: "3a02f065d365a9dbe85d296440aa220c"
server_key_iv: "b152378c130435d084890a4b620a343f"
server_key_version: "encryptionkey@0"
I20260808 07:04:31.601110 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "46598b70251c4a9bbfd3a2cc6d40d206"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-nfb5"
server_key: "3a02f065d365a9dbe85d296440aa220c"
server_key_iv: "b152378c130435d084890a4b620a343f"
server_key_version: "encryptionkey@0"
I20260808 07:04:31.602404 22549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:31.603230 22565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:31.603451 22549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:31.603545 22549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "46598b70251c4a9bbfd3a2cc6d40d206"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-nfb5"
server_key: "3a02f065d365a9dbe85d296440aa220c"
server_key_iv: "b152378c130435d084890a4b620a343f"
server_key_version: "encryptionkey@0"
I20260808 07:04:31.603602 22549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:31.611092 22549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:31.612900 22549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:31.613008 22549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:31.616844 22549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39967
I20260808 07:04:31.616912 22641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39967 every 8 connection(s)
I20260808 07:04:31.617228 22549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:31.618116 22642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:31.618870 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22549
I20260808 07:04:31.619069 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:31.619278 25301 external_mini_cluster.cc:1428] Setting key 1028da4ff94f83f1c277034e6a800826
I20260808 07:04:31.621219 22642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206: Bootstrap starting.
I20260808 07:04:31.621845 22642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:31.622140 22642 log.cc:824] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:31.622843 22642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206: No bootstrap required, opened a new log
I20260808 07:04:31.624714 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:31.619813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37075 (local address 127.0.0.1:39967)
0808 07:04:31.620029 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:31.620038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:31.620073 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:04:31.620512 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:31.620518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:31.620564 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:31.620745 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:31.620967 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.621906 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.622775 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.623256 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.623387 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:31.624103 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:31.624108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:31.624109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:31.624277 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:31.624306 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:31.624317 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:31.624396 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:31.624478 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 07:04:31.624545 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260808 07:04:31.624822 22642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260808 07:04:31.624987 22642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:31.625030 22642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46598b70251c4a9bbfd3a2cc6d40d206, State: Initialized, Role: FOLLOWER
I20260808 07:04:31.625180 22642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260808 07:04:31.625279 22642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:31.625327 22642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:31.625383 22642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:31.625694 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:31.626121 22642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260808 07:04:31.626251 22642 leader_election.cc:302] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46598b70251c4a9bbfd3a2cc6d40d206; no voters:
I20260808 07:04:31.626430 22642 leader_election.cc:288] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:31.626500 22648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:31.626652 22648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [term 1 LEADER]: Becoming Leader. State: Replica: 46598b70251c4a9bbfd3a2cc6d40d206, State: Running, Role: LEADER
I20260808 07:04:31.626696 22642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:31.626806 22648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260808 07:04:31.627141 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260808 07:04:31.627221 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:31.627382 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46598b70251c4a9bbfd3a2cc6d40d206. Latest consensus state: current_term: 1 leader_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46598b70251c4a9bbfd3a2cc6d40d206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260808 07:04:31.627444 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:31.627524 22660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:31.628178 22660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:31.631069 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:31.627570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48285 (local address 127.0.0.1:39967)
0808 07:04:31.627605 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:31.627606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:31.628061 (+ 455us) server_negotiation.cc:407] Connection header received
0808 07:04:31.628185 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:31.628186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:31.628198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:31.628259 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:31.628490 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.629076 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.629753 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.630048 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.630122 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:31.630769 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:31.630771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:31.630771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:31.630776 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:31.630792 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:31.630797 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:31.630863 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:31.630958 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:04:31.630984 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:04:31.632294 22660 catalog_manager.cc:1347] Generated new cluster ID: 8bf491fc3a994f47b631d0454cf606c2
I20260808 07:04:31.632364 22660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:31.644204 22660 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:31.644727 22660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:31.647112 22660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46598b70251c4a9bbfd3a2cc6d40d206: Generated new TSK 0
I20260808 07:04:31.647286 22660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:31.656442 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22549
2026-08-08T07:04:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:31 dist-test-slave-nfb5 krb5kdc[22689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:31 dist-test-slave-nfb5 krb5kdc[22689](info): set up 2 sockets
Aug 08 07:04:31 dist-test-slave-nfb5 krb5kdc[22689](info): commencing operation
krb5kdc: starting...
W20260808 07:04:32.866784 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:32 dist-test-slave-nfb5 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:32.989789 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:33.072145 22705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:33.072373 22705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:33.072398 22705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:33.073575 22705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:33.073621 22705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:33.073635 22705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:33.073648 22705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:33.073704 22705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:33.075358 22705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:33.075618 22705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:33.075867 22705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:33.080765 22715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.080811 22711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.080796 22712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.081009 22705 server_base.cc:1034] running on GCE node
I20260808 07:04:33.081308 22705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:33.081635 22705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:33.082809 22705 hybrid_clock.cc:648] HybridClock initialized: now 1786172673082779 us; error 49 us; skew 500 ppm
Aug 08 07:04:33 dist-test-slave-nfb5 krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:33.085469 22705 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:33.085924 22705 webserver.cc:460] Webserver started at http://127.0.0.1:42443/ using document root <none> and password file <none>
I20260808 07:04:33.086175 22705 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:33.086247 22705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:33.086404 22705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:33.087687 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "b5419e23d5fd429eb50dfa25e989ffb5"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-nfb5"
server_key: "02c3f0b30a2b7e5964a6c3b0abfddfcf"
server_key_iv: "db4f272db62a0c3f2db4533111f732c5"
server_key_version: "encryptionkey@0"
I20260808 07:04:33.088086 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "b5419e23d5fd429eb50dfa25e989ffb5"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-nfb5"
server_key: "02c3f0b30a2b7e5964a6c3b0abfddfcf"
server_key_iv: "db4f272db62a0c3f2db4533111f732c5"
server_key_version: "encryptionkey@0"
I20260808 07:04:33.089532 22705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:33.090354 22726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.090551 22705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:33.090657 22705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "b5419e23d5fd429eb50dfa25e989ffb5"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-nfb5"
server_key: "02c3f0b30a2b7e5964a6c3b0abfddfcf"
server_key_iv: "db4f272db62a0c3f2db4533111f732c5"
server_key_version: "encryptionkey@0"
I20260808 07:04:33.090783 22705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:33.094112 22705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:33.096168 22705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:33.096324 22705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:33.100389 22705 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45903
I20260808 07:04:33.100421 22803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45903 every 8 connection(s)
I20260808 07:04:33.100761 22705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:33.101733 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:33.104140 22805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5: Bootstrap starting.
I20260808 07:04:33.104708 22805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:33.104998 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22705
I20260808 07:04:33.105026 22805 log.cc:824] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:33.105139 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:33.105338 25301 external_mini_cluster.cc:1428] Setting key 28e9da99200154734e8ce99a81d7f5e5
I20260808 07:04:33.105804 22805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5: No bootstrap required, opened a new log
I20260808 07:04:33.107929 22805 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } }
I20260808 07:04:33.108058 22805 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:33.108081 22805 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5419e23d5fd429eb50dfa25e989ffb5, State: Initialized, Role: FOLLOWER
I20260808 07:04:33.108225 22805 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } }
I20260808 07:04:33.108337 22805 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:33.108430 22805 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:33.108484 22805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:33.108671 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.105865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40061 (local address 127.0.0.1:45903)
0808 07:04:33.106057 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:33.106066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:33.106102 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:04:33.107088 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:33.107095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:33.107121 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:33.107300 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:33.107990 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:33.107994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:33.107994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:33.107998 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:33.108047 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:33.108073 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:33.108136 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:33.108268 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:04:33.108311 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260808 07:04:33.109153 22805 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } }
I20260808 07:04:33.109279 22805 leader_election.cc:302] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5419e23d5fd429eb50dfa25e989ffb5; no voters:
I20260808 07:04:33.109441 22805 leader_election.cc:288] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:33.109460 22810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:33.109580 22810 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [term 1 LEADER]: Becoming Leader. State: Replica: b5419e23d5fd429eb50dfa25e989ffb5, State: Running, Role: LEADER
I20260808 07:04:33.109637 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:33.109687 22805 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:33.109735 22810 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } }
I20260808 07:04:33.110077 22810 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5419e23d5fd429eb50dfa25e989ffb5. Latest consensus state: current_term: 1 leader_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } } }
I20260808 07:04:33.110157 22810 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:33.110313 22811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5419e23d5fd429eb50dfa25e989ffb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45903 } } }
I20260808 07:04:33.110385 22811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:33.110498 22820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:33.111269 22820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:33.112893 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.111408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34775 (local address 127.0.0.1:45903)
0808 07:04:33.111441 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:33.111443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:33.111692 (+ 249us) server_negotiation.cc:407] Connection header received
0808 07:04:33.111805 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:33.111806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:33.111811 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:33.111867 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:33.112699 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:33.112700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:33.112700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:33.112704 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:33.112716 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:33.112720 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:33.112769 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:33.112815 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:04:33.112829 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:04:33.113497 22820 catalog_manager.cc:1347] Generated new cluster ID: b7217098754b48ca80b802aaf6a997cb
I20260808 07:04:33.113552 22820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:33.126690 22820 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:33.126853 22820 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:33.132166 22820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5419e23d5fd429eb50dfa25e989ffb5: Generated new TSK 0
I20260808 07:04:33.132391 22820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:33.137398 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22705
2026-08-08T07:04:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1483 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:33 dist-test-slave-nfb5 krb5kdc[22846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:33 dist-test-slave-nfb5 krb5kdc[22846](info): set up 2 sockets
Aug 08 07:04:33 dist-test-slave-nfb5 krb5kdc[22846](info): commencing operation
krb5kdc: starting...
W20260808 07:04:34.431268 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:34.556305 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:34.637790 22863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:34.638036 22863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:34.638077 22863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:34.639263 22863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:34.639343 22863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:34.639371 22863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:34.639400 22863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:34.639430 22863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:34.640923 22863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:34.641214 22863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:34.641453 22863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:34.646356 22868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:34.646474 22872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:34.646567 22863 server_base.cc:1034] running on GCE node
W20260808 07:04:34.646377 22869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:34.646884 22863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:34.647213 22863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:34.648386 22863 hybrid_clock.cc:648] HybridClock initialized: now 1786172674648366 us; error 42 us; skew 500 ppm
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:34.650996 22863 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:34.651432 22863 webserver.cc:460] Webserver started at http://127.0.0.1:40675/ using document root <none> and password file <none>
I20260808 07:04:34.651656 22863 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:34.651723 22863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:34.651883 22863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:34.652916 22863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "703ad7239532428aa19f0c1c109f5ba6"
format_stamp: "Formatted at 2026-08-08 07:04:34 on dist-test-slave-nfb5"
server_key: "578e3b87cefc3799a6605654c81bdc79"
server_key_iv: "b142a74f5bd261fc24d013ffcc432e2b"
server_key_version: "encryptionkey@0"
I20260808 07:04:34.653287 22863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "703ad7239532428aa19f0c1c109f5ba6"
format_stamp: "Formatted at 2026-08-08 07:04:34 on dist-test-slave-nfb5"
server_key: "578e3b87cefc3799a6605654c81bdc79"
server_key_iv: "b142a74f5bd261fc24d013ffcc432e2b"
server_key_version: "encryptionkey@0"
I20260808 07:04:34.654619 22863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:34.655433 22880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.655608 22863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.655718 22863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "703ad7239532428aa19f0c1c109f5ba6"
format_stamp: "Formatted at 2026-08-08 07:04:34 on dist-test-slave-nfb5"
server_key: "578e3b87cefc3799a6605654c81bdc79"
server_key_iv: "b142a74f5bd261fc24d013ffcc432e2b"
server_key_version: "encryptionkey@0"
I20260808 07:04:34.655795 22863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:34.678745 22863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:34.681248 22863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:34.681422 22863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:34.685689 22863 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33269
I20260808 07:04:34.685725 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33269 every 8 connection(s)
I20260808 07:04:34.686096 22863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:34.687074 22957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:34.689368 22957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6: Bootstrap starting.
I20260808 07:04:34.689895 22957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:34.690145 22957 log.cc:824] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:34.690819 22957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6: No bootstrap required, opened a new log
I20260808 07:04:34.691444 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 22863
I20260808 07:04:34.691591 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:34.691783 25301 external_mini_cluster.cc:1428] Setting key 7da411ade4d61db38c4a7c7ee231f653
I20260808 07:04:34.693102 22957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } }
I20260808 07:04:34.693276 22957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:34.693315 22957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703ad7239532428aa19f0c1c109f5ba6, State: Initialized, Role: FOLLOWER
I20260808 07:04:34.693521 22957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } }
I20260808 07:04:34.693608 22957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:34.693632 22957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:34.693696 22957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:34.694783 22957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } }
I20260808 07:04:34.694892 22957 leader_election.cc:302] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 703ad7239532428aa19f0c1c109f5ba6; no voters:
I20260808 07:04:34.695063 22957 leader_election.cc:288] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:34.695120 22965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:34.695291 22957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:34.695299 22965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 703ad7239532428aa19f0c1c109f5ba6, State: Running, Role: LEADER
I20260808 07:04:34.695526 22965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } }
I20260808 07:04:34.695950 22967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 703ad7239532428aa19f0c1c109f5ba6. Latest consensus state: current_term: 1 leader_uuid: "703ad7239532428aa19f0c1c109f5ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } } }
I20260808 07:04:34.696147 22967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:34.696079 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "703ad7239532428aa19f0c1c109f5ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703ad7239532428aa19f0c1c109f5ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33269 } } }
I20260808 07:04:34.696360 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:34.696461 22970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:34.697039 22970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:34.698752 22970 catalog_manager.cc:1347] Generated new cluster ID: 405378df4b4d43418533cd6ac2084f14
I20260808 07:04:34.698817 22970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:04:34.704677 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.692357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:42881 (local address 127.0.0.1:33269)
0808 07:04:34.692557 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:34.692568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:34.692621 (+ 53us) server_negotiation.cc:407] Connection header received
0808 07:04:34.693060 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:34.693068 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:34.693145 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:34.693299 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:34.694083 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.695572 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.696644 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.697259 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.697364 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.700377 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:34.700381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:34.700381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:34.700393 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:34.702434 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.702874 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.702877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.702878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.703042 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.703339 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.703340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.703340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.703649 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:34.703826 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:34.704339 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 07:04:34.704467 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260808 07:04:34.705966 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:34.725091 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.707464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:53809 (local address 127.0.0.1:33269)
0808 07:04:34.707499 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:34.707501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:34.717427 (+ 9926us) server_negotiation.cc:407] Connection header received
0808 07:04:34.717493 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:34.717493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:34.717504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:34.717565 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:34.718210 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.719296 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.720312 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.720654 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.720714 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.722163 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:34.722164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:34.722165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:34.722171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:34.723824 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.724286 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.724287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.724287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.724343 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.724581 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.724582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.724582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.724694 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:34.724781 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:34.724942 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:04:34.725004 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:04:34.737957 22970 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:34.738478 22970 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:34.741930 22970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 703ad7239532428aa19f0c1c109f5ba6: Generated new TSK 0
I20260808 07:04:34.742136 22970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:34.748446 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 22863
W20260808 07:04:34.760865 22843 connection.cc:535] client connection to 127.0.0.1:33269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33269): Connection reset by peer (error 104)
2026-08-08T07:04:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1612 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[23002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[23002](info): set up 2 sockets
Aug 08 07:04:34 dist-test-slave-nfb5 krb5kdc[23002](info): commencing operation
krb5kdc: starting...
W20260808 07:04:35.979828 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:36 dist-test-slave-nfb5 krb5kdc[23002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:36.107903 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:36.189384 23019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:36.189627 23019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:36.189666 23019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:36.190789 23019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:36.190866 23019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:36.190896 23019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:36.190940 23019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:36.190975 23019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:36.192445 23019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:36.192727 23019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:36.192960 23019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:36.198101 23019 server_base.cc:1034] running on GCE node
W20260808 07:04:36.198076 23025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:36.198094 23024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:36.198068 23028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:36.198618 23019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:36.198975 23019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:36.200145 23019 hybrid_clock.cc:648] HybridClock initialized: now 1786172676200124 us; error 42 us; skew 500 ppm
Aug 08 07:04:36 dist-test-slave-nfb5 krb5kdc[23002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:36.206025 23019 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:36.206593 23019 webserver.cc:460] Webserver started at http://127.0.0.1:40811/ using document root <none> and password file <none>
I20260808 07:04:36.206939 23019 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:36.207017 23019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:36.207165 23019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:36.208485 23019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c"
format_stamp: "Formatted at 2026-08-08 07:04:36 on dist-test-slave-nfb5"
server_key: "fe4b02bb4f0a2f517b35b4a86fe65a9c"
server_key_iv: "1409f1f1f07a00a513583046917302c3"
server_key_version: "encryptionkey@0"
I20260808 07:04:36.208937 23019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c"
format_stamp: "Formatted at 2026-08-08 07:04:36 on dist-test-slave-nfb5"
server_key: "fe4b02bb4f0a2f517b35b4a86fe65a9c"
server_key_iv: "1409f1f1f07a00a513583046917302c3"
server_key_version: "encryptionkey@0"
I20260808 07:04:36.210605 23019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:36.211431 23039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:36.211627 23019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:36.211750 23019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c"
format_stamp: "Formatted at 2026-08-08 07:04:36 on dist-test-slave-nfb5"
server_key: "fe4b02bb4f0a2f517b35b4a86fe65a9c"
server_key_iv: "1409f1f1f07a00a513583046917302c3"
server_key_version: "encryptionkey@0"
I20260808 07:04:36.211830 23019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:36.225663 23019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:36.227581 23019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:36.227746 23019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:36.231678 23019 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35289
I20260808 07:04:36.231709 23113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35289 every 8 connection(s)
I20260808 07:04:36.232052 23019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:36.232669 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23019
I20260808 07:04:36.232797 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:36.233027 25301 external_mini_cluster.cc:1428] Setting key d46128916520057b511f9e8245cc70b6
I20260808 07:04:36.233160 23114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:36.236515 23114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: Bootstrap starting.
I20260808 07:04:36.237147 23114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:36.237525 23114 log.cc:824] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:36.238314 23114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: No bootstrap required, opened a new log
I20260808 07:04:36.238653 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:36.233591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:47021 (local address 127.0.0.1:35289)
0808 07:04:36.233780 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:36.233788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:36.233823 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:04:36.234297 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:36.234304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:36.234350 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:36.234457 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:36.234681 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:36.235673 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:36.236763 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:36.237146 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:36.237250 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:36.238039 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:36.238046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:36.238046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:36.238208 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:36.238236 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:36.238247 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:36.238313 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:36.238467 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:04:36.238501 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260808 07:04:36.239817 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:36.240687 23114 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260808 07:04:36.240871 23114 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:36.240911 23114 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b25066c81444f0bf6ac28c7b3fdf4c, State: Initialized, Role: FOLLOWER
I20260808 07:04:36.241025 23114 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260808 07:04:36.241101 23114 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:36.241241 23114 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:36.241302 23114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:36.241851 23114 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260808 07:04:36.241974 23114 leader_election.cc:302] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b25066c81444f0bf6ac28c7b3fdf4c; no voters:
I20260808 07:04:36.242153 23114 leader_election.cc:288] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:36.242203 23128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:36.242360 23128 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [term 1 LEADER]: Becoming Leader. State: Replica: e3b25066c81444f0bf6ac28c7b3fdf4c, State: Running, Role: LEADER
I20260808 07:04:36.242427 23114 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:36.242520 23128 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260808 07:04:36.243456 23129 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260808 07:04:36.243489 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b25066c81444f0bf6ac28c7b3fdf4c. Latest consensus state: current_term: 1 leader_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b25066c81444f0bf6ac28c7b3fdf4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260808 07:04:36.243608 23129 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:36.243620 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c [sys.catalog]: This master's current role is: LEADER
W20260808 07:04:36.244442 23149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:36.244489 23149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:36.244555 23143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:36.244700 23143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:36.246078 23143 catalog_manager.cc:1347] Generated new cluster ID: 6f7808840529498699797f84d9a75bd3
I20260808 07:04:36.246136 23143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:36.247649 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:36.241200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:39741 (local address 127.0.0.1:35289)
0808 07:04:36.241238 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:36.241240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:36.241333 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:04:36.241404 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:36.241405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:36.241414 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:36.241469 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:36.241617 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:36.242182 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:36.243324 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:36.243645 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:36.243694 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:36.247401 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:36.247403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:36.247403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:36.247408 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:36.247422 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:36.247426 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:36.247483 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:36.247564 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:04:36.247574 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:04:36.270530 23143 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:36.271149 23143 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:36.276077 23143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b25066c81444f0bf6ac28c7b3fdf4c: Generated new TSK 0
I20260808 07:04:36.276289 23143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:36.280603 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23019
W20260808 07:04:36.294157 22999 connection.cc:535] client connection to 127.0.0.1:35289 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35289): Connection reset by peer (error 104)
2026-08-08T07:04:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:36 dist-test-slave-nfb5 krb5kdc[23157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:36 dist-test-slave-nfb5 krb5kdc[23157](info): set up 2 sockets
Aug 08 07:04:36 dist-test-slave-nfb5 krb5kdc[23157](info): commencing operation
krb5kdc: starting...
W20260808 07:04:37.537235 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:37 dist-test-slave-nfb5 krb5kdc[23157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:37.658617 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:37.739619 23175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:37.739857 23175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:37.739898 23175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:37.741055 23175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:37.741127 23175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:37.741153 23175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:37.741185 23175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:37.741211 23175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:37.742662 23175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:37.742962 23175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:37.743203 23175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:37.748118 23181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:37.748133 23183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:37.748373 23189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:37.748351 23175 server_base.cc:1034] running on GCE node
I20260808 07:04:37.748636 23175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:37.748880 23175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:37.750017 23175 hybrid_clock.cc:648] HybridClock initialized: now 1786172677750008 us; error 28 us; skew 500 ppm
Aug 08 07:04:37 dist-test-slave-nfb5 krb5kdc[23157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:37.752493 23175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:37.752955 23175 webserver.cc:460] Webserver started at http://127.0.0.1:35935/ using document root <none> and password file <none>
I20260808 07:04:37.753202 23175 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:37.753273 23175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:37.753427 23175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:37.754442 23175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "72b5292e111e4c6b856ece5a1f45e217"
format_stamp: "Formatted at 2026-08-08 07:04:37 on dist-test-slave-nfb5"
server_key: "37eb937ada3bef64b29af8825f61477f"
server_key_iv: "1a6fdd57204967d56dd8d2f9dd560edc"
server_key_version: "encryptionkey@0"
I20260808 07:04:37.754801 23175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "72b5292e111e4c6b856ece5a1f45e217"
format_stamp: "Formatted at 2026-08-08 07:04:37 on dist-test-slave-nfb5"
server_key: "37eb937ada3bef64b29af8825f61477f"
server_key_iv: "1a6fdd57204967d56dd8d2f9dd560edc"
server_key_version: "encryptionkey@0"
I20260808 07:04:37.756165 23175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:37.756901 23198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:37.757114 23175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:37.757217 23175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "72b5292e111e4c6b856ece5a1f45e217"
format_stamp: "Formatted at 2026-08-08 07:04:37 on dist-test-slave-nfb5"
server_key: "37eb937ada3bef64b29af8825f61477f"
server_key_iv: "1a6fdd57204967d56dd8d2f9dd560edc"
server_key_version: "encryptionkey@0"
I20260808 07:04:37.757298 23175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:37.760608 23175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:37.762905 23175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:37.763118 23175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:37.767391 23175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38951
I20260808 07:04:37.767438 23269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38951 every 8 connection(s)
I20260808 07:04:37.767773 23175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:37.768741 23271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:37.770900 23271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217: Bootstrap starting.
I20260808 07:04:37.771561 23271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:37.771811 23271 log.cc:824] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:37.772464 23271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217: No bootstrap required, opened a new log
I20260808 07:04:37.773231 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23175
I20260808 07:04:37.773375 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:37.773550 25301 external_mini_cluster.cc:1428] Setting key 1dc1b950f011c54e98b0d2a8754b6d55
I20260808 07:04:37.774595 23271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } }
I20260808 07:04:37.774765 23271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:37.774803 23271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b5292e111e4c6b856ece5a1f45e217, State: Initialized, Role: FOLLOWER
I20260808 07:04:37.775048 23271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } }
I20260808 07:04:37.775161 23271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:37.775210 23271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:37.775278 23271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:37.775988 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:37.774123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:49055 (local address 127.0.0.1:38951)
0808 07:04:37.774337 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:37.774346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:37.774383 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:04:37.774551 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:37.774559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:37.774589 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:37.774748 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:37.775481 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:37.775485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:37.775485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:37.775490 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:37.775535 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:37.775549 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:37.775624 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:37.775681 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:04:37.775739 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260808 07:04:37.776432 23271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } }
I20260808 07:04:37.776538 23271 leader_election.cc:302] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72b5292e111e4c6b856ece5a1f45e217; no voters:
I20260808 07:04:37.776706 23271 leader_election.cc:288] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:37.776768 23279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:37.776942 23271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:37.776991 23279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [term 1 LEADER]: Becoming Leader. State: Replica: 72b5292e111e4c6b856ece5a1f45e217, State: Running, Role: LEADER
I20260808 07:04:37.777048 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:37.777091 23279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } }
I20260808 07:04:37.777449 23281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b5292e111e4c6b856ece5a1f45e217. Latest consensus state: current_term: 1 leader_uuid: "72b5292e111e4c6b856ece5a1f45e217" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } } }
I20260808 07:04:37.777451 23280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72b5292e111e4c6b856ece5a1f45e217" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b5292e111e4c6b856ece5a1f45e217" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38951 } } }
I20260808 07:04:37.777838 23280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:37.778029 23281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:37.778472 23288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:37.779042 23288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:37.779767 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:37.778766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:33437 (local address 127.0.0.1:38951)
0808 07:04:37.778799 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:37.778800 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:37.778808 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:37.778829 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:37.778830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:37.778835 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:37.778885 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:37.779527 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:37.779528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:37.779528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:37.779533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:37.779547 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:37.779551 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:37.779608 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:37.779664 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:04:37.779684 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:04:37.783038 23288 catalog_manager.cc:1347] Generated new cluster ID: 32e503e3d1634fec8d55705451659daa
I20260808 07:04:37.783099 23288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:37.797247 23288 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:37.797422 23288 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:37.804401 23288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b5292e111e4c6b856ece5a1f45e217: Generated new TSK 0
I20260808 07:04:37.804577 23288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:37.808833 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23175
2026-08-08T07:04:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:37 dist-test-slave-nfb5 krb5kdc[23312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:37 dist-test-slave-nfb5 krb5kdc[23312](info): set up 2 sockets
Aug 08 07:04:37 dist-test-slave-nfb5 krb5kdc[23312](info): commencing operation
krb5kdc: starting...
W20260808 07:04:39.058830 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:39.181260 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:39.264122 23327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:39.264320 23327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:39.264343 23327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:39.265400 23327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:39.265434 23327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:39.265450 23327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:39.265461 23327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:39.265475 23327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:39.266908 23327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:39.267125 23327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:39.267287 23327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:39.271955 23337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:39.271972 23336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:39.271996 23327 server_base.cc:1034] running on GCE node
W20260808 07:04:39.271976 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:39.272373 23327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:39.272617 23327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:39.273784 23327 hybrid_clock.cc:648] HybridClock initialized: now 1786172679273763 us; error 39 us; skew 500 ppm
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:39.276373 23327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:39.276858 23327 webserver.cc:460] Webserver started at http://127.0.0.1:46411/ using document root <none> and password file <none>
I20260808 07:04:39.277092 23327 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:39.277159 23327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:39.277318 23327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:39.278339 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "69d25b9fe15445ec9b16324e342319cc"
format_stamp: "Formatted at 2026-08-08 07:04:39 on dist-test-slave-nfb5"
server_key: "80dad75733e166754fcc1887fa891460"
server_key_iv: "9245ffec096aa8955df57b8c37eeb608"
server_key_version: "encryptionkey@0"
I20260808 07:04:39.278748 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "69d25b9fe15445ec9b16324e342319cc"
format_stamp: "Formatted at 2026-08-08 07:04:39 on dist-test-slave-nfb5"
server_key: "80dad75733e166754fcc1887fa891460"
server_key_iv: "9245ffec096aa8955df57b8c37eeb608"
server_key_version: "encryptionkey@0"
I20260808 07:04:39.280201 23327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:39.281023 23352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:39.281200 23327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:39.281302 23327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "69d25b9fe15445ec9b16324e342319cc"
format_stamp: "Formatted at 2026-08-08 07:04:39 on dist-test-slave-nfb5"
server_key: "80dad75733e166754fcc1887fa891460"
server_key_iv: "9245ffec096aa8955df57b8c37eeb608"
server_key_version: "encryptionkey@0"
I20260808 07:04:39.281385 23327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:39.308853 23327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:39.311486 23327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:39.311728 23327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:39.315915 23327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33717
I20260808 07:04:39.315948 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33717 every 8 connection(s)
I20260808 07:04:39.316323 23327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:39.317189 23426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:39.319550 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc: Bootstrap starting.
I20260808 07:04:39.320158 23426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:39.320550 23426 log.cc:824] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:39.321496 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc: No bootstrap required, opened a new log
I20260808 07:04:39.323160 23426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 07:04:39.323302 23426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:39.323324 23426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d25b9fe15445ec9b16324e342319cc, State: Initialized, Role: FOLLOWER
I20260808 07:04:39.323457 23426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 07:04:39.323537 23426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:39.323560 23426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:39.323588 23426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:39.324082 23426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 07:04:39.324167 23426 leader_election.cc:302] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69d25b9fe15445ec9b16324e342319cc; no voters:
I20260808 07:04:39.324296 23426 leader_election.cc:288] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:39.324363 23429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:39.324517 23429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [term 1 LEADER]: Becoming Leader. State: Replica: 69d25b9fe15445ec9b16324e342319cc, State: Running, Role: LEADER
I20260808 07:04:39.324527 23426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:39.324620 23429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 07:04:39.325038 23431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69d25b9fe15445ec9b16324e342319cc. Latest consensus state: current_term: 1 leader_uuid: "69d25b9fe15445ec9b16324e342319cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260808 07:04:39.325132 23431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:39.325242 23430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69d25b9fe15445ec9b16324e342319cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d25b9fe15445ec9b16324e342319cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260808 07:04:39.325356 23430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:39.325724 23439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:39.326216 23439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:39.326330 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23327
I20260808 07:04:39.326469 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:39.326663 25301 external_mini_cluster.cc:1428] Setting key aaf0fd7d19cb4c5f65e632add0a33e4a
I20260808 07:04:39.328711 23439 catalog_manager.cc:1347] Generated new cluster ID: 1eb94e2ea7f449ccab08dfcab557fca0
I20260808 07:04:39.328770 23439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:04:39.338639 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:39.327362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:52819 (local address 127.0.0.1:33717)
0808 07:04:39.327537 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:39.327545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:39.327583 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:04:39.327671 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:39.327677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:39.327720 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:39.327869 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:39.328614 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:39.330057 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:39.331227 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:39.331850 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:39.333704 (+ 1854us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:39.334864 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:39.334868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:39.334868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:39.334878 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:39.336790 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:39.337249 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:39.337252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:39.337252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:39.337350 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:39.337603 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:39.337604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:39.337604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:39.337884 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:39.338008 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:39.338190 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 07:04:39.338403 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260808 07:04:39.339799 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:39.343663 23439 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:39.344321 23439 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:39.347004 23439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69d25b9fe15445ec9b16324e342319cc: Generated new TSK 0
I20260808 07:04:39.347175 23439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:39.348845 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:39.341401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:49367 (local address 127.0.0.1:33717)
0808 07:04:39.341430 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:39.341432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:39.341440 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:39.341508 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:39.341509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:39.341530 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:39.341574 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:39.342208 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:39.342953 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:39.343958 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:39.344288 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:39.344367 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:39.345868 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:39.345870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:39.345870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:39.345877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:39.347661 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:39.348050 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:39.348051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:39.348051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:39.348113 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:39.348350 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:39.348351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:39.348351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:39.348479 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:39.348582 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:39.348691 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:04:39.348754 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 07:04:39.350951 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23327
W20260808 07:04:39.365545 23309 connection.cc:535] client connection to 127.0.0.1:33717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33717): Connection reset by peer (error 104)
2026-08-08T07:04:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23467](info): set up 2 sockets
Aug 08 07:04:39 dist-test-slave-nfb5 krb5kdc[23467](info): commencing operation
krb5kdc: starting...
W20260808 07:04:40.617144 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:40 dist-test-slave-nfb5 krb5kdc[23467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:40.744059 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:40.829505 23485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:40.829720 23485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:40.829743 23485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:40.830960 23485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:40.831019 23485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:40.831035 23485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:40.831081 23485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:40.831100 23485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:40.832682 23485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:40.832939 23485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:40.833170 23485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:40.838011 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:40.838042 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:40.837986 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:40.838304 23485 server_base.cc:1034] running on GCE node
I20260808 07:04:40.838620 23485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:40.838992 23485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:40.840165 23485 hybrid_clock.cc:648] HybridClock initialized: now 1786172680840152 us; error 28 us; skew 500 ppm
Aug 08 07:04:40 dist-test-slave-nfb5 krb5kdc[23467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:40.842671 23485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:40.843169 23485 webserver.cc:460] Webserver started at http://127.0.0.1:43631/ using document root <none> and password file <none>
I20260808 07:04:40.843413 23485 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:40.843484 23485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:40.843622 23485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:40.844671 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c21247545ba246c9b96661f312c2cd5d"
format_stamp: "Formatted at 2026-08-08 07:04:40 on dist-test-slave-nfb5"
server_key: "64d5d9773b72e2afa82d88f2437d1b7d"
server_key_iv: "a69819a6c5a41373d14a41283532da55"
server_key_version: "encryptionkey@0"
I20260808 07:04:40.845047 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c21247545ba246c9b96661f312c2cd5d"
format_stamp: "Formatted at 2026-08-08 07:04:40 on dist-test-slave-nfb5"
server_key: "64d5d9773b72e2afa82d88f2437d1b7d"
server_key_iv: "a69819a6c5a41373d14a41283532da55"
server_key_version: "encryptionkey@0"
I20260808 07:04:40.846424 23485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:04:40.847234 23503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:40.847426 23485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:40.847527 23485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c21247545ba246c9b96661f312c2cd5d"
format_stamp: "Formatted at 2026-08-08 07:04:40 on dist-test-slave-nfb5"
server_key: "64d5d9773b72e2afa82d88f2437d1b7d"
server_key_iv: "a69819a6c5a41373d14a41283532da55"
server_key_version: "encryptionkey@0"
I20260808 07:04:40.847613 23485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:40.862080 23485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:40.864032 23485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:40.864203 23485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:40.868548 23485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35065
I20260808 07:04:40.868587 23573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35065 every 8 connection(s)
I20260808 07:04:40.868939 23485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:40.869210 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23485
I20260808 07:04:40.869360 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:40.869581 25301 external_mini_cluster.cc:1428] Setting key 4efff35d1158c8858207a2d869573157
I20260808 07:04:40.869997 23575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:40.873595 23575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: Bootstrap starting.
I20260808 07:04:40.874306 23575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:40.874637 23575 log.cc:824] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:40.875531 23575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: No bootstrap required, opened a new log
I20260808 07:04:40.876221 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:40.870169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:39535 (local address 127.0.0.1:35065)
0808 07:04:40.870407 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:40.870417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:40.870450 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:04:40.871241 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:40.871250 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:40.871337 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:40.871492 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:40.871662 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:40.873066 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:40.873954 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:40.874595 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:40.874715 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:40.875521 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:40.875525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:40.875525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:40.875745 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:40.875782 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:40.875796 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:40.875886 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:40.875982 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:04:40.876008 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
I20260808 07:04:40.877393 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:40.877936 23575 raft_consensus.cc:348] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } }
I20260808 07:04:40.878111 23575 raft_consensus.cc:374] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:40.878157 23575 raft_consensus.cc:729] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21247545ba246c9b96661f312c2cd5d, State: Initialized, Role: FOLLOWER
I20260808 07:04:40.878279 23575 consensus_queue.cc:260] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } }
I20260808 07:04:40.878355 23575 raft_consensus.cc:388] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:40.878387 23575 raft_consensus.cc:482] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:40.878444 23575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:40.879220 23575 raft_consensus.cc:504] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } }
I20260808 07:04:40.879339 23575 leader_election.cc:302] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c21247545ba246c9b96661f312c2cd5d; no voters:
I20260808 07:04:40.879514 23575 leader_election.cc:288] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:40.879554 23590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:40.879710 23590 raft_consensus.cc:686] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [term 1 LEADER]: Becoming Leader. State: Replica: c21247545ba246c9b96661f312c2cd5d, State: Running, Role: LEADER
I20260808 07:04:40.879763 23575 sys_catalog.cc:564] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:40.879817 23590 consensus_queue.cc:237] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } }
I20260808 07:04:40.880101 23590 sys_catalog.cc:455] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c21247545ba246c9b96661f312c2cd5d. Latest consensus state: current_term: 1 leader_uuid: "c21247545ba246c9b96661f312c2cd5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } } }
I20260808 07:04:40.880172 23590 sys_catalog.cc:458] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:40.880113 23592 sys_catalog.cc:455] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c21247545ba246c9b96661f312c2cd5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21247545ba246c9b96661f312c2cd5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35065 } } }
I20260808 07:04:40.880344 23592 sys_catalog.cc:458] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d [sys.catalog]: This master's current role is: LEADER
W20260808 07:04:40.881664 23610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:40.881721 23610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:40.882040 23596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:40.882232 23596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:40.882494 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:40.879025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:47047 (local address 127.0.0.1:35065)
0808 07:04:40.879069 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:40.879071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:40.879178 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:04:40.879248 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:40.879249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:40.879257 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:40.879313 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:40.879542 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:40.880222 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:40.881092 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:40.881406 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:40.881479 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:40.882232 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:40.882233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:40.882233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:40.882238 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:40.882251 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:40.882255 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:40.882301 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:40.882419 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:04:40.882431 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:40.883886 23596 catalog_manager.cc:1347] Generated new cluster ID: c0d8503e21a04c438e83e125819f3d5c
I20260808 07:04:40.883941 23596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:40.911720 23596 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:40.912281 23596 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:40.915014 23596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c21247545ba246c9b96661f312c2cd5d: Generated new TSK 0
I20260808 07:04:40.915206 23596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:40.921172 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23485
W20260808 07:04:40.934036 23463 connection.cc:535] client connection to 127.0.0.1:35065 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35065): Connection reset by peer (error 104)
2026-08-08T07:04:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:40 dist-test-slave-nfb5 krb5kdc[23617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:40 dist-test-slave-nfb5 krb5kdc[23617](info): set up 2 sockets
Aug 08 07:04:40 dist-test-slave-nfb5 krb5kdc[23617](info): commencing operation
krb5kdc: starting...
W20260808 07:04:42.171705 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:42 dist-test-slave-nfb5 krb5kdc[23617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:04:42.297660 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:42.379060 23635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:42.379283 23635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:42.379320 23635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:42.380456 23635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:42.380527 23635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:42.380551 23635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:42.380585 23635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:42.380610 23635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:42.382092 23635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:42.382349 23635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:42.382573 23635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:42.387573 23647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:42.387599 23644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:42.387732 23643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:42.387804 23635 server_base.cc:1034] running on GCE node
I20260808 07:04:42.388015 23635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:42.388325 23635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:42.389495 23635 hybrid_clock.cc:648] HybridClock initialized: now 1786172682389471 us; error 45 us; skew 500 ppm
Aug 08 07:04:42 dist-test-slave-nfb5 krb5kdc[23617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:42.395309 23635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:04:42.395753 23635 webserver.cc:460] Webserver started at http://127.0.0.1:46081/ using document root <none> and password file <none>
I20260808 07:04:42.395983 23635 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:42.396052 23635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:42.396211 23635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:42.397311 23635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "56e316a6ed7c4417bb6a915f97eb5681"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-nfb5"
server_key: "cd90332a1a1a859320fa0402a4aa9f74"
server_key_iv: "9014240a4f58c46169addf8197b9e8d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.397680 23635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "56e316a6ed7c4417bb6a915f97eb5681"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-nfb5"
server_key: "cd90332a1a1a859320fa0402a4aa9f74"
server_key_iv: "9014240a4f58c46169addf8197b9e8d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.399044 23635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:42.399853 23657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:42.400040 23635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:42.400153 23635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "56e316a6ed7c4417bb6a915f97eb5681"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-nfb5"
server_key: "cd90332a1a1a859320fa0402a4aa9f74"
server_key_iv: "9014240a4f58c46169addf8197b9e8d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.400226 23635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:42.403599 23635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:42.405458 23635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:42.405615 23635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:42.409786 23635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35635
I20260808 07:04:42.409818 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35635 every 8 connection(s)
I20260808 07:04:42.410149 23635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:42.411181 23727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:42.412106 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23635
I20260808 07:04:42.412269 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:42.412461 25301 external_mini_cluster.cc:1428] Setting key e7ba19003030afb90ad02e288e80b55e
I20260808 07:04:42.414659 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681: Bootstrap starting.
I20260808 07:04:42.415019 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:42.413030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:33071 (local address 127.0.0.1:35635)
0808 07:04:42.413225 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:42.413232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:42.413257 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:04:42.413663 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:42.413669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:42.413693 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:42.413809 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:42.414495 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:42.414498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:42.414499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:42.414503 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:42.414539 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:42.414549 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:42.414615 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:42.414706 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:04:42.414775 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260808 07:04:42.415501 23727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:42.415830 23727 log.cc:824] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:42.415979 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:42.416664 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681: No bootstrap required, opened a new log
I20260808 07:04:42.418560 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:42.417585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:40997 (local address 127.0.0.1:35635)
0808 07:04:42.417639 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:42.417640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:42.417694 (+ 54us) server_negotiation.cc:407] Connection header received
0808 07:04:42.417764 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:42.417765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:42.417768 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:42.417810 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:42.418351 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:42.418352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:42.418352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:42.418356 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:42.418367 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:42.418370 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:42.418415 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:42.418468 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:04:42.418477 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:04:42.419414 23727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } }
I20260808 07:04:42.419569 23727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:42.419617 23727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e316a6ed7c4417bb6a915f97eb5681, State: Initialized, Role: FOLLOWER
I20260808 07:04:42.419744 23727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } }
I20260808 07:04:42.419843 23727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:42.419888 23727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:42.419939 23727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:42.420480 23727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } }
I20260808 07:04:42.420603 23727 leader_election.cc:302] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56e316a6ed7c4417bb6a915f97eb5681; no voters:
I20260808 07:04:42.420800 23727 leader_election.cc:288] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:42.420843 23739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:42.421058 23739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [term 1 LEADER]: Becoming Leader. State: Replica: 56e316a6ed7c4417bb6a915f97eb5681, State: Running, Role: LEADER
I20260808 07:04:42.421068 23727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:42.421259 23739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } }
I20260808 07:04:42.421641 23740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56e316a6ed7c4417bb6a915f97eb5681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } } }
I20260808 07:04:42.421657 23741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56e316a6ed7c4417bb6a915f97eb5681. Latest consensus state: current_term: 1 leader_uuid: "56e316a6ed7c4417bb6a915f97eb5681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e316a6ed7c4417bb6a915f97eb5681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35635 } } }
I20260808 07:04:42.421823 23741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:42.422039 23740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:42.422241 23752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:42.422729 23752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:42.424230 23752 catalog_manager.cc:1347] Generated new cluster ID: 15834836268d4868968e9591daffabc5
I20260808 07:04:42.424295 23752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:42.442586 23752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:42.442801 23752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:42.445652 23752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56e316a6ed7c4417bb6a915f97eb5681: Generated new TSK 0
I20260808 07:04:42.445840 23752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:42.452502 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23635
2026-08-08T07:04:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:42 dist-test-slave-nfb5 krb5kdc[23768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:42 dist-test-slave-nfb5 krb5kdc[23768](info): set up 2 sockets
Aug 08 07:04:42 dist-test-slave-nfb5 krb5kdc[23768](info): commencing operation
krb5kdc: starting...
W20260808 07:04:43.663529 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:43 dist-test-slave-nfb5 krb5kdc[23768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:43.786098 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43121
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:46309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:43.867556 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:43.867823 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:43.867865 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:43.869011 23782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:43.869099 23782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:43.869122 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:43.869143 23782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:43.869175 23782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:43.870659 23782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:46309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:43121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:43.870970 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:43.871201 23782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:43.876067 23789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.876044 23788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.876044 23794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:43.876380 23782 server_base.cc:1034] running on GCE node
I20260808 07:04:43.876605 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:43.876837 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:43.878010 23782 hybrid_clock.cc:648] HybridClock initialized: now 1786172683878008 us; error 53 us; skew 500 ppm
Aug 08 07:04:43 dist-test-slave-nfb5 krb5kdc[23768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:43.880491 23782 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:43.880918 23782 webserver.cc:460] Webserver started at http://127.24.181.126:41671/ using document root <none> and password file <none>
I20260808 07:04:43.881157 23782 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:43.881227 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:43.881381 23782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:43.882402 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "b4d36ab15eba425db974e286240cf935"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-nfb5"
server_key: "11d06523d5a998b4d797c0b38f96fcf3"
server_key_iv: "529758568b88ed3d99a383662c34bced"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.882763 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "b4d36ab15eba425db974e286240cf935"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-nfb5"
server_key: "11d06523d5a998b4d797c0b38f96fcf3"
server_key_iv: "529758568b88ed3d99a383662c34bced"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.884153 23782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:43.884912 23806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.885089 23782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.885190 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "b4d36ab15eba425db974e286240cf935"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-nfb5"
server_key: "11d06523d5a998b4d797c0b38f96fcf3"
server_key_iv: "529758568b88ed3d99a383662c34bced"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.885269 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:43.906324 23782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:43.908250 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:43.908434 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:43.912401 23782 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:43121
I20260808 07:04:43.912456 23883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:43121 every 8 connection(s)
I20260808 07:04:43.912817 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:43.913738 23884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:43.916074 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935: Bootstrap starting.
I20260808 07:04:43.916666 23884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:43.916971 23884 log.cc:824] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:43.917696 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935: No bootstrap required, opened a new log
I20260808 07:04:43.919425 23884 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } }
I20260808 07:04:43.919559 23884 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:43.919605 23884 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d36ab15eba425db974e286240cf935, State: Initialized, Role: FOLLOWER
I20260808 07:04:43.919783 23884 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } }
I20260808 07:04:43.919867 23884 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:43.919919 23884 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:43.919971 23884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:43.920858 23884 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } }
I20260808 07:04:43.920933 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23782
I20260808 07:04:43.920981 23884 leader_election.cc:302] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4d36ab15eba425db974e286240cf935; no voters:
I20260808 07:04:43.921065 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:43.921167 23884 leader_election.cc:288] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:43.921200 23887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:43.921288 25301 external_mini_cluster.cc:1428] Setting key 3bfa4f09ff83b29efdbdea99a5bcd6d9
I20260808 07:04:43.921396 23887 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [term 1 LEADER]: Becoming Leader. State: Replica: b4d36ab15eba425db974e286240cf935, State: Running, Role: LEADER
I20260808 07:04:43.921427 23884 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:43.921474 23887 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } }
I20260808 07:04:43.921864 23888 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4d36ab15eba425db974e286240cf935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } } }
I20260808 07:04:43.921937 23888 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:43.922112 23889 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4d36ab15eba425db974e286240cf935. Latest consensus state: current_term: 1 leader_uuid: "b4d36ab15eba425db974e286240cf935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d36ab15eba425db974e286240cf935" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 43121 } } }
I20260808 07:04:43.922166 23889 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:43.922482 23898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:43.923049 23898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:43.927408 23898 catalog_manager.cc:1347] Generated new cluster ID: 7f8b3f3ad1fb4d24bc5aea266ce13ae2
I20260808 07:04:43.927466 23898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:04:43 dist-test-slave-nfb5 krb5kdc[23768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:43.933909 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.921942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59405 (local address 127.24.181.126:43121)
0808 07:04:43.922147 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.922157 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.922414 (+ 257us) server_negotiation.cc:407] Connection header received
0808 07:04:43.922703 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.922711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.922789 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.922982 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.923792 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.925146 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.925872 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.926495 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.926608 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.929497 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.929501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.929502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.929512 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.932115 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.932590 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.932593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.932593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.932706 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.932959 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.932959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.932960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.933256 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.933395 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.933578 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.933677 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260808 07:04:43.935199 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:43.940379 23898 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:43.940883 23898 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:43.944021 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.936676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60951 (local address 127.24.181.126:43121)
0808 07:04:43.936712 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.936713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.936809 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:04:43.936853 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.936854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.936864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.936931 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.937490 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.938260 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.939262 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.939598 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.939668 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.941242 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.941243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.941243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.941250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.942747 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.943192 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.943193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.943194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.943249 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.943487 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.943488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.943488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.943602 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.943703 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.943873 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.943936 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:04:43.946842 23898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4d36ab15eba425db974e286240cf935: Generated new TSK 0
I20260808 07:04:43.947050 23898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:43.951978 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23782
W20260808 07:04:43.966120 23764 connection.cc:535] client connection to 127.24.181.126:43121 recv error: Network error: failed to read from TLS socket (remote: 127.24.181.126:43121): Connection reset by peer (error 104)
2026-08-08T07:04:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:44 dist-test-slave-nfb5 krb5kdc[23926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:44 dist-test-slave-nfb5 krb5kdc[23926](info): set up 2 sockets
Aug 08 07:04:44 dist-test-slave-nfb5 krb5kdc[23926](info): commencing operation
krb5kdc: starting...
W20260808 07:04:45.180001 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:45 dist-test-slave-nfb5 krb5kdc[23926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:45.301276 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37437
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:45.383581 23946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:45.383798 23946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:45.383821 23946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:45.384917 23946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:45.384954 23946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:45.384968 23946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:45.384981 23946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:45.384994 23946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:45.386435 23946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:37437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:45.386623 23946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:45.386804 23946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:45.391682 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.391691 23957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.391701 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.391963 23946 server_base.cc:1034] running on GCE node
I20260808 07:04:45.392191 23946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:45.392477 23946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:45.393646 23946 hybrid_clock.cc:648] HybridClock initialized: now 1786172685393643 us; error 43 us; skew 500 ppm
Aug 08 07:04:45 dist-test-slave-nfb5 krb5kdc[23926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:45.396309 23946 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:45.396739 23946 webserver.cc:460] Webserver started at http://127.24.181.126:39209/ using document root <none> and password file <none>
I20260808 07:04:45.396987 23946 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:45.397037 23946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:45.397166 23946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:45.398183 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "ba2a66b9c73c4406915667c2994b34fb"
format_stamp: "Formatted at 2026-08-08 07:04:45 on dist-test-slave-nfb5"
server_key: "89d60eba58509699893314779592d279"
server_key_iv: "59378b3cfdfa150461edeb10e1581de9"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.398535 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "ba2a66b9c73c4406915667c2994b34fb"
format_stamp: "Formatted at 2026-08-08 07:04:45 on dist-test-slave-nfb5"
server_key: "89d60eba58509699893314779592d279"
server_key_iv: "59378b3cfdfa150461edeb10e1581de9"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.399959 23946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:45.400749 23965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.400930 23946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:45.401038 23946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "ba2a66b9c73c4406915667c2994b34fb"
format_stamp: "Formatted at 2026-08-08 07:04:45 on dist-test-slave-nfb5"
server_key: "89d60eba58509699893314779592d279"
server_key_iv: "59378b3cfdfa150461edeb10e1581de9"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.401113 23946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:45.415522 23946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:45.417799 23946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:45.417963 23946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:45.422201 23946 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:37437
I20260808 07:04:45.422242 24036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:37437 every 8 connection(s)
I20260808 07:04:45.422617 23946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:45.423595 24037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:45.425908 24037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb: Bootstrap starting.
I20260808 07:04:45.426013 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 23946
I20260808 07:04:45.426162 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:45.426367 25301 external_mini_cluster.cc:1428] Setting key a3fc2490727abcb3a3193e5dbfb8f853
I20260808 07:04:45.426658 24037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:45.427045 24037 log.cc:824] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:45.427821 24037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb: No bootstrap required, opened a new log
I20260808 07:04:45.430235 24037 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } }
I20260808 07:04:45.430416 24037 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:45.430460 24037 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2a66b9c73c4406915667c2994b34fb, State: Initialized, Role: FOLLOWER
I20260808 07:04:45.430585 24037 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } }
I20260808 07:04:45.430656 24037 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:45.430687 24037 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:45.430732 24037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:45.431490 24037 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } }
I20260808 07:04:45.431618 24037 leader_election.cc:302] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba2a66b9c73c4406915667c2994b34fb; no voters:
I20260808 07:04:45.431824 24037 leader_election.cc:288] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:45.431860 24046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:45.432032 24046 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [term 1 LEADER]: Becoming Leader. State: Replica: ba2a66b9c73c4406915667c2994b34fb, State: Running, Role: LEADER
I20260808 07:04:45.432121 24046 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } }
I20260808 07:04:45.432173 24037 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:45.432461 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba2a66b9c73c4406915667c2994b34fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } } }
I20260808 07:04:45.432749 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:45.432816 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.426953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39731 (local address 127.24.181.126:37437)
0808 07:04:45.427147 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:45.427156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:45.427192 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:04:45.427769 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:45.427779 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:45.427852 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:45.428020 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:45.428232 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:45.429613 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:45.430419 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:45.431079 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:45.431202 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:45.431942 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:45.431947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:45.431947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:45.432182 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:45.432254 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:45.432270 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:45.432385 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:45.432553 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:04:45.432581 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260808 07:04:45.432981 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba2a66b9c73c4406915667c2994b34fb. Latest consensus state: current_term: 1 leader_uuid: "ba2a66b9c73c4406915667c2994b34fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2a66b9c73c4406915667c2994b34fb" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37437 } } }
I20260808 07:04:45.433106 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:45.433111 24061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:45.433718 24061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:45.433962 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:45.438710 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.435603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60027 (local address 127.24.181.126:37437)
0808 07:04:45.435643 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:45.435645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:45.435710 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:04:45.435738 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:45.435739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:45.435748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:45.435817 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:45.435945 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:45.436659 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:45.437374 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:45.437696 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:45.437774 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:45.438465 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:45.438467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:45.438467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:45.438472 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:45.438488 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:45.438493 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:45.438560 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:45.438623 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:04:45.438635 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:45.439232 24061 catalog_manager.cc:1347] Generated new cluster ID: 1019519097b04573bbbe6d03180474be
I20260808 07:04:45.440482 24061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:45.449272 24061 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:45.450047 24061 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:45.455125 24061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba2a66b9c73c4406915667c2994b34fb: Generated new TSK 0
I20260808 07:04:45.455317 24061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:45.460980 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 23946
W20260808 07:04:45.472349 23924 connection.cc:535] client connection to 127.24.181.126:37437 recv error: Network error: failed to read from TLS socket (remote: 127.24.181.126:37437): Connection reset by peer (error 104)
2026-08-08T07:04:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:45 dist-test-slave-nfb5 krb5kdc[24085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:45 dist-test-slave-nfb5 krb5kdc[24085](info): set up 2 sockets
Aug 08 07:04:45 dist-test-slave-nfb5 krb5kdc[24085](info): commencing operation
krb5kdc: starting...
W20260808 07:04:46.669441 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:46 dist-test-slave-nfb5 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:46.789803 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:35951
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:35951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:46.874171 24102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:46.874392 24102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:46.874413 24102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:46.875618 24102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:46.875662 24102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:46.875677 24102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:46.875691 24102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:46.875754 24102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:46.877250 24102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:35951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:35951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:46.877501 24102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:46.877741 24102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:46.882591 24112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:46.882671 24102 server_base.cc:1034] running on GCE node
W20260808 07:04:46.882637 24108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:46.882603 24109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:46.883091 24102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:46.883391 24102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:46.884539 24102 hybrid_clock.cc:648] HybridClock initialized: now 1786172686884519 us; error 50 us; skew 500 ppm
Aug 08 07:04:46 dist-test-slave-nfb5 krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172686, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:46.887121 24102 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:46.887507 24102 webserver.cc:460] Webserver started at http://127.24.181.126:41307/ using document root <none> and password file <none>
I20260808 07:04:46.887702 24102 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:46.887744 24102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:46.887835 24102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:46.888828 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "7c0230e2948341ed8be04dbacb2192cd"
format_stamp: "Formatted at 2026-08-08 07:04:46 on dist-test-slave-nfb5"
server_key: "9e5a4fae9e26b0b0a0cb1baece2d20ca"
server_key_iv: "92a5e4cf495a8b9bd63c5b9ddbb5c4b3"
server_key_version: "encryptionkey@0"
I20260808 07:04:46.889155 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "7c0230e2948341ed8be04dbacb2192cd"
format_stamp: "Formatted at 2026-08-08 07:04:46 on dist-test-slave-nfb5"
server_key: "9e5a4fae9e26b0b0a0cb1baece2d20ca"
server_key_iv: "92a5e4cf495a8b9bd63c5b9ddbb5c4b3"
server_key_version: "encryptionkey@0"
I20260808 07:04:46.890480 24102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:46.891189 24121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:46.891367 24102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:46.891479 24102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "7c0230e2948341ed8be04dbacb2192cd"
format_stamp: "Formatted at 2026-08-08 07:04:46 on dist-test-slave-nfb5"
server_key: "9e5a4fae9e26b0b0a0cb1baece2d20ca"
server_key_iv: "92a5e4cf495a8b9bd63c5b9ddbb5c4b3"
server_key_version: "encryptionkey@0"
I20260808 07:04:46.891556 24102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:46.894819 24102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:46.896622 24102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:46.896785 24102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:46.900943 24102 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:35951
I20260808 07:04:46.900987 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:35951 every 8 connection(s)
I20260808 07:04:46.901315 24102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:46.902308 24192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:46.904495 24192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd: Bootstrap starting.
I20260808 07:04:46.905015 24192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:46.905297 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24102
I20260808 07:04:46.905386 24192 log.cc:824] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:46.905437 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:46.905628 25301 external_mini_cluster.cc:1428] Setting key b4706584b40c9a9a8ae13184e4070ae0
I20260808 07:04:46.906164 24192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd: No bootstrap required, opened a new log
I20260808 07:04:46.908250 24192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } }
I20260808 07:04:46.908388 24192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:46.908412 24192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0230e2948341ed8be04dbacb2192cd, State: Initialized, Role: FOLLOWER
I20260808 07:04:46.908748 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.906144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44051 (local address 127.24.181.126:35951)
0808 07:04:46.906345 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:46.906354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:46.906392 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:04:46.907415 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:46.907422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:46.907448 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:46.907597 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:46.908296 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:46.908307 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:46.908307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:46.908311 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:46.908350 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:46.908360 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:46.908441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:46.908550 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:04:46.908592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260808 07:04:46.908547 24192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } }
I20260808 07:04:46.909024 24192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:46.909080 24192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:46.909132 24192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:46.909761 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:46.909893 24192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } }
I20260808 07:04:46.910020 24192 leader_election.cc:302] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c0230e2948341ed8be04dbacb2192cd; no voters:
I20260808 07:04:46.910203 24192 leader_election.cc:288] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:46.910250 24200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:46.910420 24200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [term 1 LEADER]: Becoming Leader. State: Replica: 7c0230e2948341ed8be04dbacb2192cd, State: Running, Role: LEADER
I20260808 07:04:46.910471 24192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:46.910607 24200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } }
I20260808 07:04:46.911044 24200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c0230e2948341ed8be04dbacb2192cd. Latest consensus state: current_term: 1 leader_uuid: "7c0230e2948341ed8be04dbacb2192cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } } }
I20260808 07:04:46.911057 24204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c0230e2948341ed8be04dbacb2192cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c0230e2948341ed8be04dbacb2192cd" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 35951 } } }
I20260808 07:04:46.911360 24204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:46.911644 24200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:46.911791 24225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:46.912360 24225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:46.912648 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.911396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57903 (local address 127.24.181.126:35951)
0808 07:04:46.911433 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:46.911435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:46.911549 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:04:46.911657 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:46.911658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:46.911664 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:46.911698 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:46.912436 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:46.912437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:46.912437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:46.912441 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:46.912453 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:46.912457 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:46.912502 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:46.912555 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:04:46.912569 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:04:46.913938 24225 catalog_manager.cc:1347] Generated new cluster ID: ad35f71056824483b2c357499a36c28c
I20260808 07:04:46.913990 24225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:46.922269 24225 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:46.922413 24225 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:46.925875 24225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c0230e2948341ed8be04dbacb2192cd: Generated new TSK 0
I20260808 07:04:46.926059 24225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:46.933231 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24102
2026-08-08T07:04:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:46 dist-test-slave-nfb5 krb5kdc[24237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:46 dist-test-slave-nfb5 krb5kdc[24237](info): set up 2 sockets
Aug 08 07:04:46 dist-test-slave-nfb5 krb5kdc[24237](info): commencing operation
krb5kdc: starting...
W20260808 07:04:48.158461 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:48.279912 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44977
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:44977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:48.360530 24251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:48.360745 24251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:48.360771 24251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:48.361931 24251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:48.361975 24251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:48.361990 24251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:48.362002 24251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:48.362015 24251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:48.363547 24251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:44977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:44977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:48.363798 24251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:48.364056 24251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:48.369000 24257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:48.369074 24256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:48.369221 24261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:48.369200 24251 server_base.cc:1034] running on GCE node
I20260808 07:04:48.369465 24251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:48.369763 24251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:48.370971 24251 hybrid_clock.cc:648] HybridClock initialized: now 1786172688370945 us; error 45 us; skew 500 ppm
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172688, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:48.377130 24251 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:48.377573 24251 webserver.cc:460] Webserver started at http://127.24.181.126:44681/ using document root <none> and password file <none>
I20260808 07:04:48.377835 24251 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:48.377884 24251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:48.378053 24251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:48.379164 24251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "dcb307f91199424f99e9b2680b93ea68"
format_stamp: "Formatted at 2026-08-08 07:04:48 on dist-test-slave-nfb5"
server_key: "8a0b1f0b686b2c0eace3e81a703641a8"
server_key_iv: "3d01b4ac4c4debe3b6dffe9d15917633"
server_key_version: "encryptionkey@0"
I20260808 07:04:48.379525 24251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "dcb307f91199424f99e9b2680b93ea68"
format_stamp: "Formatted at 2026-08-08 07:04:48 on dist-test-slave-nfb5"
server_key: "8a0b1f0b686b2c0eace3e81a703641a8"
server_key_iv: "3d01b4ac4c4debe3b6dffe9d15917633"
server_key_version: "encryptionkey@0"
I20260808 07:04:48.380844 24251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:48.381580 24269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:48.381757 24251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:48.381866 24251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "dcb307f91199424f99e9b2680b93ea68"
format_stamp: "Formatted at 2026-08-08 07:04:48 on dist-test-slave-nfb5"
server_key: "8a0b1f0b686b2c0eace3e81a703641a8"
server_key_iv: "3d01b4ac4c4debe3b6dffe9d15917633"
server_key_version: "encryptionkey@0"
I20260808 07:04:48.381942 24251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:48.401396 24251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:48.403793 24251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:48.403960 24251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:48.408218 24251 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:44977
I20260808 07:04:48.408254 24338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:44977 every 8 connection(s)
I20260808 07:04:48.408604 24251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:48.409473 24340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:48.411759 24340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: Bootstrap starting.
I20260808 07:04:48.412276 24340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:48.412532 24340 log.cc:824] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:48.413196 24340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: No bootstrap required, opened a new log
I20260808 07:04:48.414570 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24251
I20260808 07:04:48.414691 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:48.414865 25301 external_mini_cluster.cc:1428] Setting key a02135214241062486c9c2305a1c6b82
I20260808 07:04:48.414822 24340 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } }
I20260808 07:04:48.415031 24340 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:48.415082 24340 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb307f91199424f99e9b2680b93ea68, State: Initialized, Role: FOLLOWER
I20260808 07:04:48.415226 24340 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } }
I20260808 07:04:48.415310 24340 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:48.415354 24340 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:48.415400 24340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:48.416239 24340 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } }
I20260808 07:04:48.416334 24340 leader_election.cc:302] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcb307f91199424f99e9b2680b93ea68; no voters:
I20260808 07:04:48.416512 24340 leader_election.cc:288] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:48.416561 24351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:48.416767 24351 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [term 1 LEADER]: Becoming Leader. State: Replica: dcb307f91199424f99e9b2680b93ea68, State: Running, Role: LEADER
I20260808 07:04:48.416775 24340 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:48.418247 24351 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } }
I20260808 07:04:48.418877 24352 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcb307f91199424f99e9b2680b93ea68. Latest consensus state: current_term: 1 leader_uuid: "dcb307f91199424f99e9b2680b93ea68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } } }
I20260808 07:04:48.419023 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcb307f91199424f99e9b2680b93ea68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb307f91199424f99e9b2680b93ea68" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 44977 } } }
I20260808 07:04:48.419102 24352 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:48.419123 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68 [sys.catalog]: This master's current role is: LEADER
W20260808 07:04:48.419788 24370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:48.419843 24370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:48.419898 24372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:48.420058 24372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:48.421631 24372 catalog_manager.cc:1347] Generated new cluster ID: e891a864de42442da56209b5d67e6f2c
I20260808 07:04:48.421676 24372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:48.425508 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.415404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35393 (local address 127.24.181.126:44977)
0808 07:04:48.415602 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:48.415612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:48.415642 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:04:48.415752 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:48.415757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:48.415795 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:48.415931 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:48.416732 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:48.417814 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:48.418768 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:48.419432 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:48.419549 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:48.422128 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:48.422132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:48.422132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:48.422144 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:48.423907 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:48.424299 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:48.424302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:48.424302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:48.424406 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:48.424638 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:48.424639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:48.424639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:48.424868 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:48.425091 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:48.425157 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:04:48.425302 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260808 07:04:48.426376 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:48.433849 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.427745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42705 (local address 127.24.181.126:44977)
0808 07:04:48.427781 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:48.427783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:48.427912 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:04:48.427982 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:48.427983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:48.427993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:48.428046 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:48.428527 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:48.429004 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:48.429719 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:48.429939 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:48.430019 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:48.431263 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:48.431264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:48.431265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:48.431272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:48.432699 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:48.433051 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:48.433052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:48.433052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:48.433109 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:48.433315 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:48.433316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:48.433316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:48.433437 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:48.433546 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:48.433648 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:04:48.433727 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:04:48.436287 24372 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:48.436781 24372 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:48.441849 24372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcb307f91199424f99e9b2680b93ea68: Generated new TSK 0
I20260808 07:04:48.442037 24372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:48.444943 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24251
2026-08-08T07:04:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24384](info): set up 2 sockets
Aug 08 07:04:48 dist-test-slave-nfb5 krb5kdc[24384](info): commencing operation
krb5kdc: starting...
W20260808 07:04:49.666646 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:49 dist-test-slave-nfb5 krb5kdc[24384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:49.789068 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45227
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:45227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:49.870054 24406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:49.870260 24406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:49.870283 24406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:49.871387 24406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:49.871424 24406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:49.871438 24406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:49.871451 24406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:49.871464 24406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:49.872859 24406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:45227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:45227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:49.873041 24406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:49.873199 24406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:49.877996 24420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:49.878023 24422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:49.878021 24418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:49.878155 24406 server_base.cc:1034] running on GCE node
I20260808 07:04:49.878461 24406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:49.878757 24406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:49.879935 24406 hybrid_clock.cc:648] HybridClock initialized: now 1786172689879925 us; error 52 us; skew 500 ppm
Aug 08 07:04:49 dist-test-slave-nfb5 krb5kdc[24384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172689, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:49.882504 24406 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:49.883008 24406 webserver.cc:460] Webserver started at http://127.24.181.126:45269/ using document root <none> and password file <none>
I20260808 07:04:49.883251 24406 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:49.883332 24406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:49.883468 24406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:49.884497 24406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "e56c5ad00cef4cefa1efa379d37ef4fe"
format_stamp: "Formatted at 2026-08-08 07:04:49 on dist-test-slave-nfb5"
server_key: "c98fb9b82f02914867b2b0cc1a37388f"
server_key_iv: "e196a20e9298429eb806afe3906360b1"
server_key_version: "encryptionkey@0"
I20260808 07:04:49.884864 24406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "e56c5ad00cef4cefa1efa379d37ef4fe"
format_stamp: "Formatted at 2026-08-08 07:04:49 on dist-test-slave-nfb5"
server_key: "c98fb9b82f02914867b2b0cc1a37388f"
server_key_iv: "e196a20e9298429eb806afe3906360b1"
server_key_version: "encryptionkey@0"
I20260808 07:04:49.886215 24406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:49.886988 24434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:49.887171 24406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:49.887274 24406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "e56c5ad00cef4cefa1efa379d37ef4fe"
format_stamp: "Formatted at 2026-08-08 07:04:49 on dist-test-slave-nfb5"
server_key: "c98fb9b82f02914867b2b0cc1a37388f"
server_key_iv: "e196a20e9298429eb806afe3906360b1"
server_key_version: "encryptionkey@0"
I20260808 07:04:49.887359 24406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:49.895910 24406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:49.897617 24406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:49.897789 24406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:49.901893 24406 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:45227
I20260808 07:04:49.901932 24506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:45227 every 8 connection(s)
I20260808 07:04:49.902261 24406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:49.903201 24507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:49.903743 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24406
I20260808 07:04:49.903885 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:49.904072 25301 external_mini_cluster.cc:1428] Setting key e3a593920528bb624d989ae6301d12a5
I20260808 07:04:49.905974 24507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: Bootstrap starting.
I20260808 07:04:49.906607 24507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:49.906958 24507 log.cc:824] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:49.907716 24507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: No bootstrap required, opened a new log
I20260808 07:04:49.909793 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:49.904598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33867 (local address 127.24.181.126:45227)
0808 07:04:49.904775 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:49.904782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:49.904806 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:04:49.905198 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:49.905203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:49.905245 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:49.905351 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:49.905588 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:49.906485 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:49.907599 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:49.908180 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:49.908293 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:49.908978 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:49.908982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:49.908982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:49.909182 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:49.909220 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:49.909236 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:49.909313 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:49.909450 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:04:49.909528 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260808 07:04:49.910036 24507 raft_consensus.cc:348] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } }
I20260808 07:04:49.910203 24507 raft_consensus.cc:374] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:49.910240 24507 raft_consensus.cc:729] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56c5ad00cef4cefa1efa379d37ef4fe, State: Initialized, Role: FOLLOWER
I20260808 07:04:49.910378 24507 consensus_queue.cc:260] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } }
I20260808 07:04:49.910466 24507 raft_consensus.cc:388] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:49.910509 24507 raft_consensus.cc:482] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:49.910540 24507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:49.910768 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:49.911265 24507 raft_consensus.cc:504] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } }
I20260808 07:04:49.911384 24507 leader_election.cc:302] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e56c5ad00cef4cefa1efa379d37ef4fe; no voters:
I20260808 07:04:49.911548 24507 leader_election.cc:288] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:49.911608 24515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:49.911777 24515 raft_consensus.cc:686] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [term 1 LEADER]: Becoming Leader. State: Replica: e56c5ad00cef4cefa1efa379d37ef4fe, State: Running, Role: LEADER
I20260808 07:04:49.911794 24507 sys_catalog.cc:564] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:49.911924 24515 consensus_queue.cc:237] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } }
I20260808 07:04:49.913136 24518 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader e56c5ad00cef4cefa1efa379d37ef4fe. Latest consensus state: current_term: 1 leader_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } } }
I20260808 07:04:49.913219 24518 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:49.913342 24517 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56c5ad00cef4cefa1efa379d37ef4fe" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 45227 } } }
I20260808 07:04:49.913453 24517 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe [sys.catalog]: This master's current role is: LEADER
W20260808 07:04:49.913724 24544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:49.913789 24544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:49.913887 24545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:49.914052 24545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:49.915575 24545 catalog_manager.cc:1347] Generated new cluster ID: eef73670a1c84f70a661f94c8dbe35d4
I20260808 07:04:49.915645 24545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:49.915961 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:49.912366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40459 (local address 127.24.181.126:45227)
0808 07:04:49.912403 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:49.912406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:49.912698 (+ 292us) server_negotiation.cc:407] Connection header received
0808 07:04:49.912775 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:49.912776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:49.912786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:49.912833 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:49.913011 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:49.913739 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:49.914707 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:49.914896 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:49.915028 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:49.915699 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:49.915701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:49.915701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:49.915706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:49.915720 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:49.915723 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:49.915774 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:49.915873 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:04:49.915896 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:04:49.918998 24545 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:49.919710 24545 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:49.935141 24545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e56c5ad00cef4cefa1efa379d37ef4fe: Generated new TSK 0
I20260808 07:04:49.935319 24545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:49.939065 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24406
2026-08-08T07:04:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:49 dist-test-slave-nfb5 krb5kdc[24552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:49 dist-test-slave-nfb5 krb5kdc[24552](info): set up 2 sockets
Aug 08 07:04:49 dist-test-slave-nfb5 krb5kdc[24552](info): commencing operation
krb5kdc: starting...
W20260808 07:04:51.177068 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:51 dist-test-slave-nfb5 krb5kdc[24552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:51.300215 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37007
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:37007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:51.381067 24572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:51.381316 24572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:51.381354 24572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:51.382490 24572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:51.382566 24572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:51.382589 24572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:51.382609 24572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:51.382642 24572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:51.384148 24572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:37007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:37007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:51.384429 24572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:51.384662 24572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:51.389712 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:51.389739 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:51.389739 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:51.389920 24572 server_base.cc:1034] running on GCE node
I20260808 07:04:51.390304 24572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:51.390632 24572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:51.391809 24572 hybrid_clock.cc:648] HybridClock initialized: now 1786172691391799 us; error 41 us; skew 500 ppm
Aug 08 07:04:51 dist-test-slave-nfb5 krb5kdc[24552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:51.394284 24572 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:51.394788 24572 webserver.cc:460] Webserver started at http://127.24.181.126:43525/ using document root <none> and password file <none>
I20260808 07:04:51.395066 24572 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:51.395136 24572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:51.395292 24572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:51.396327 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "c0eba9f5f087427e8ec894df7b404868"
format_stamp: "Formatted at 2026-08-08 07:04:51 on dist-test-slave-nfb5"
server_key: "32f7442b15fde336cd38a00ec6dafbb2"
server_key_iv: "898059f7e827ddeafd50921bdfad105e"
server_key_version: "encryptionkey@0"
I20260808 07:04:51.396692 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "c0eba9f5f087427e8ec894df7b404868"
format_stamp: "Formatted at 2026-08-08 07:04:51 on dist-test-slave-nfb5"
server_key: "32f7442b15fde336cd38a00ec6dafbb2"
server_key_iv: "898059f7e827ddeafd50921bdfad105e"
server_key_version: "encryptionkey@0"
I20260808 07:04:51.398000 24572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:51.398674 24593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:51.398865 24572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:51.398984 24572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "c0eba9f5f087427e8ec894df7b404868"
format_stamp: "Formatted at 2026-08-08 07:04:51 on dist-test-slave-nfb5"
server_key: "32f7442b15fde336cd38a00ec6dafbb2"
server_key_iv: "898059f7e827ddeafd50921bdfad105e"
server_key_version: "encryptionkey@0"
I20260808 07:04:51.399068 24572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:51.402290 24572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:51.404129 24572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:51.404282 24572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:51.408310 24572 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:37007
I20260808 07:04:51.408349 24671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:37007 every 8 connection(s)
I20260808 07:04:51.408664 24572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:51.409595 24672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:51.411859 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868: Bootstrap starting.
I20260808 07:04:51.412379 24672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:51.412627 24672 log.cc:824] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:51.413385 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868: No bootstrap required, opened a new log
I20260808 07:04:51.414832 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24572
I20260808 07:04:51.414981 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:51.414906 24672 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } }
I20260808 07:04:51.415156 24672 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:51.415205 24672 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eba9f5f087427e8ec894df7b404868, State: Initialized, Role: FOLLOWER
I20260808 07:04:51.415225 25301 external_mini_cluster.cc:1428] Setting key 18dd6e013fd7c91ce7128a24ecf0d198
I20260808 07:04:51.415350 24672 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } }
I20260808 07:04:51.415444 24672 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:51.415483 24672 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:51.415536 24672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:51.416689 24672 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } }
I20260808 07:04:51.416814 24672 leader_election.cc:302] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0eba9f5f087427e8ec894df7b404868; no voters:
I20260808 07:04:51.416992 24672 leader_election.cc:288] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:51.417027 24678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:51.417218 24678 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [term 1 LEADER]: Becoming Leader. State: Replica: c0eba9f5f087427e8ec894df7b404868, State: Running, Role: LEADER
I20260808 07:04:51.417251 24672 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:51.417373 24678 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } }
I20260808 07:04:51.417572 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:51.415721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59733 (local address 127.24.181.126:37007)
0808 07:04:51.415922 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:51.415932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:51.415974 (+ 42us) server_negotiation.cc:407] Connection header received
0808 07:04:51.416112 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:51.416119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:51.416156 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:51.416332 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:51.416994 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:51.416998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:51.416998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:51.417003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:51.417049 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:51.417065 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:51.417137 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:51.417327 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:04:51.417371 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260808 07:04:51.418455 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:51.418553 24681 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0eba9f5f087427e8ec894df7b404868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } } }
I20260808 07:04:51.418651 24681 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:51.418875 24699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:51.418933 24679 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0eba9f5f087427e8ec894df7b404868. Latest consensus state: current_term: 1 leader_uuid: "c0eba9f5f087427e8ec894df7b404868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0eba9f5f087427e8ec894df7b404868" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 37007 } } }
I20260808 07:04:51.418987 24679 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:51.419529 24699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:51.421336 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:51.420099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36735 (local address 127.24.181.126:37007)
0808 07:04:51.420136 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:51.420138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:51.420264 (+ 126us) server_negotiation.cc:407] Connection header received
0808 07:04:51.420354 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:51.420355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:51.420360 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:51.420388 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:51.421029 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:51.421030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:51.421030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:04:51.421036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:51.421052 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:04:51.421057 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:51.421121 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:51.421229 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:04:51.421246 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:04:51.421589 24699 catalog_manager.cc:1347] Generated new cluster ID: 67cfb9ee18714c25a18e4f37e4e3eef5
I20260808 07:04:51.421644 24699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:51.431131 24699 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:51.431303 24699 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:51.435729 24699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0eba9f5f087427e8ec894df7b404868: Generated new TSK 0
I20260808 07:04:51.435923 24699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:51.441256 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24572
2026-08-08T07:04:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1505 ms)
[----------] 18 tests from AuthTokenIssuingTest (27406 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:51 dist-test-slave-nfb5 krb5kdc[24712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:51 dist-test-slave-nfb5 krb5kdc[24712](info): set up 2 sockets
Aug 08 07:04:51 dist-test-slave-nfb5 krb5kdc[24712](info): commencing operation
krb5kdc: starting...
W20260808 07:04:52.659286 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:52 dist-test-slave-nfb5 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:52.783981 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41163
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:52.870160 24727 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:52.870406 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:52.870436 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:52.870479 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:52.871642 24727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:52.871716 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:52.871740 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:52.871760 24727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:52.871791 24727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:52.873245 24727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:41163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:52.873531 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:52.873759 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:52.878553 24733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:52.878615 24727 server_base.cc:1034] running on GCE node
W20260808 07:04:52.878639 24738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:52.878540 24734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:52.879107 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:52.879411 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:52.880587 24727 hybrid_clock.cc:648] HybridClock initialized: now 1786172692880569 us; error 39 us; skew 500 ppm
Aug 08 07:04:52 dist-test-slave-nfb5 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:52.883255 24727 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:52.883702 24727 webserver.cc:460] Webserver started at http://127.24.181.126:33099/ using document root <none> and password file <none>
I20260808 07:04:52.883958 24727 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:52.884007 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:52.884162 24727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:52.885215 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "f307794712ff41dbb3636aee7641a649"
format_stamp: "Formatted at 2026-08-08 07:04:52 on dist-test-slave-nfb5"
server_key: "25e09e8898bd53855850ea9d1ed32952"
server_key_iv: "328a2580ff0013fc0bec33c14cfcb860"
server_key_version: "encryptionkey@0"
I20260808 07:04:52.885586 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "f307794712ff41dbb3636aee7641a649"
format_stamp: "Formatted at 2026-08-08 07:04:52 on dist-test-slave-nfb5"
server_key: "25e09e8898bd53855850ea9d1ed32952"
server_key_iv: "328a2580ff0013fc0bec33c14cfcb860"
server_key_version: "encryptionkey@0"
I20260808 07:04:52.886976 24727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:52.887691 24749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:52.887924 24727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:52.888018 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "f307794712ff41dbb3636aee7641a649"
format_stamp: "Formatted at 2026-08-08 07:04:52 on dist-test-slave-nfb5"
server_key: "25e09e8898bd53855850ea9d1ed32952"
server_key_iv: "328a2580ff0013fc0bec33c14cfcb860"
server_key_version: "encryptionkey@0"
I20260808 07:04:52.888090 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:52.901361 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:52.903792 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:52.903967 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:52.908283 24727 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:41163
I20260808 07:04:52.908360 24827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:41163 every 8 connection(s)
I20260808 07:04:52.908696 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:52.909252 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24727
I20260808 07:04:52.909382 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:52.909562 25301 external_mini_cluster.cc:1428] Setting key 0fcab4a2b29779af727ac0b734f90378
I20260808 07:04:52.909688 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:52.911366 24828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:04:52 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.125@KRBTEST.COM" created.
I20260808 07:04:52.922119 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.912023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41163 (local address 127.0.0.1:48864)
0808 07:04:52.912183 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.912232 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:52.912708 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:52.913009 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:52.913012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:52.913254 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:52.913399 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:52.913403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:52.915008 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:52.915009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:52.915676 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:52.915679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:52.916387 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:52.916388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:52.916448 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:52.917040 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:52.917048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:52.918768 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:52.920502 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:52.920508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:52.920510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:52.920767 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:52.921026 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:52.921027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:52.921027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:52.921144 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:52.921508 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:52.921513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:52.921575 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 07:04:52.921690 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:04:52.921936 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":103,"thread_start_us":88,"threads_started":1}
I20260808 07:04:52.922128 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.912218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.24.181.126:41163)
0808 07:04:52.912384 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:52.912389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:52.912426 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:04:52.912862 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:52.912864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:52.912886 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:52.912992 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:52.913506 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:52.914884 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:52.915772 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:52.916279 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:52.916413 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:52.918866 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:52.918873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:52.918873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:52.918880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:52.920433 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:52.920847 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:52.920851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:52.920851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:52.920946 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:52.921204 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:52.921205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:52.921205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:52.921426 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:52.921527 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:52.921723 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 07:04:52.921869 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260808 07:04:52.922902 24828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:52.923331 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.923209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:55974)
0808 07:04:52.923265 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.923309 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:04:52.923471 24755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
I20260808 07:04:52.923893 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.923794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:55988)
0808 07:04:52.923840 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.923864 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:04:52.923991 24828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45729: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
I20260808 07:04:52.937803 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:45729
--webserver_interface=127.24.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:52.973234 24828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } attempt: 1
I20260808 07:04:52.973752 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.973581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56002)
0808 07:04:52.973653 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.973691 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:04:52.974108 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.974011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56014)
0808 07:04:52.974056 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.974081 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:04:52.974198 24828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45729: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
W20260808 07:04:53.020102 24836 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:53.020313 24836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:53.020329 24836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:53.020350 24836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:53.021552 24836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:53.021598 24836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:53.021613 24836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:53.021626 24836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:53.021639 24836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:53.023202 24836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:45729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:53.023456 24836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:53.023687 24836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:53.028442 24848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:53.028506 24845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:53.028493 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:53.028877 24836 server_base.cc:1034] running on GCE node
I20260808 07:04:53.029083 24836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:53.029309 24836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:53.030454 24836 hybrid_clock.cc:648] HybridClock initialized: now 1786172693030443 us; error 30 us; skew 500 ppm
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:53.032970 24836 init.cc:377] Logged in from keytab as kudu/127.24.181.125@KRBTEST.COM (short username kudu)
I20260808 07:04:53.033455 24836 webserver.cc:460] Webserver started at http://127.24.181.125:34715/ using document root <none> and password file <none>
I20260808 07:04:53.033661 24836 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:53.033701 24836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:53.033790 24836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:53.034768 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data/instance:
uuid: "80dc241d985f4b629d1e673e529c0011"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "08cf86e5389653fb7742881c526eb113"
server_key_iv: "a103ae6be96b95f3325639aa0b2a8663"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.035152 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal/instance:
uuid: "80dc241d985f4b629d1e673e529c0011"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "08cf86e5389653fb7742881c526eb113"
server_key_iv: "a103ae6be96b95f3325639aa0b2a8663"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.036432 24836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:53.037122 24858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:53.037318 24836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:53.037403 24836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "80dc241d985f4b629d1e673e529c0011"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "08cf86e5389653fb7742881c526eb113"
server_key_iv: "a103ae6be96b95f3325639aa0b2a8663"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.037458 24836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:53.042342 24828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } attempt: 2
I20260808 07:04:53.042725 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.042601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56028)
0808 07:04:53.042661 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.042697 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:04:53.043119 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.043033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56030)
0808 07:04:53.043077 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.043102 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:04:53.043236 24828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45729: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45729: connect: Connection refused (error 111)
I20260808 07:04:53.044719 24836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:53.046959 24836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:53.047091 24836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:53.050993 24836 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.125:45729
I20260808 07:04:53.051015 24927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.125:45729 every 8 connection(s)
I20260808 07:04:53.051378 24836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
I20260808 07:04:53.052273 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:53.052603 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24836
I20260808 07:04:53.052696 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-1/wal/instance
I20260808 07:04:53.052942 25301 external_mini_cluster.cc:1428] Setting key 22e5accf12bc79d15d68a23678449b39
I20260808 07:04:53.053996 24930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.124@KRBTEST.COM; defaulting to no policy
I20260808 07:04:53.063256 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.054672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.24.181.126:41163)
0808 07:04:53.054721 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.054723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.054981 (+ 258us) server_negotiation.cc:407] Connection header received
0808 07:04:53.055467 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.055469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.055487 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.055571 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.056160 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.056651 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.057713 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.057937 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.058020 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.060192 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.060193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.060194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.060201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.061745 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.062168 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.062169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.062169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.062233 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.062499 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.062500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.062500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.062616 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.062694 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.063081 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.063161 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.124@KRBTEST.COM" created.
I20260808 07:04:53.063514 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.054657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41163 (local address 127.0.0.1:48868)
0808 07:04:53.054834 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.054875 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.055372 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.055637 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.055641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.055877 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.056057 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.056061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.056750 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.056751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.057615 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.057617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.058027 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.058028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.058167 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.058698 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.058706 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.060093 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.061853 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.061861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.061863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.062087 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.062312 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.062313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.062313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.062430 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.062722 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.062730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.062861 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.063060 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.063220 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260808 07:04:53.064105 24930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:53.072003 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.064366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56046)
0808 07:04:53.064415 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.064426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.064467 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.064798 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.064799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.064919 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.064965 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.064967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.065887 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.065889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.066502 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.066505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.067272 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.067273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.067385 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.067790 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.067795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.068928 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.071000 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.071001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.071001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.071160 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.071404 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.071404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.071404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.071455 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.071779 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.071780 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.071812 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.071869 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.071939 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:04:53.072144 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.064424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.24.181.125:45729)
0808 07:04:53.064628 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.064633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.064657 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:04:53.064681 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.064683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.064697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.064785 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.065042 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.065793 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.066597 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.067188 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.067271 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.069015 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.069025 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.069026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.069034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.070889 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.071257 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.071260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.071261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.071328 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.071519 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.071520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.071521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.071711 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.071797 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.071910 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.072006 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260808 07:04:53.072876 24930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:53.073241 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.073116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46356)
0808 07:04:53.073165 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.073211 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:04:53.073472 24861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
I20260808 07:04:53.073928 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.073832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46362)
0808 07:04:53.073882 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.073910 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:04:53.074085 24930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:46257: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
I20260808 07:04:53.081365 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:46257
--webserver_interface=127.24.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:53.123312 24930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } attempt: 1
I20260808 07:04:53.123793 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.123657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46364)
0808 07:04:53.123721 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.123763 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:04:53.124150 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.124052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46372)
0808 07:04:53.124098 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.124122 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:04:53.124241 24930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:46257: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
I20260808 07:04:53.134406 24828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } attempt: 3
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:53.142284 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.134776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56048)
0808 07:04:53.134855 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.134873 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.134979 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.135144 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.135145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.135329 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.135402 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.135403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.136473 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.136474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.137006 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.137008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.137555 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.137556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.137614 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.138102 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.138107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.139404 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.141337 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.141338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.141339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.141504 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.141713 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.141714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.141714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.141765 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.142052 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.142053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.142092 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.142142 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.142216 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 07:04:53.142360 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.134859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.24.181.125:45729)
0808 07:04:53.134958 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.134960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.134973 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:53.135050 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.135051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.135066 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.135138 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.135536 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.136344 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.137096 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.137472 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.137552 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.139495 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.139496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.139497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.139505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.141245 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.141578 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.141579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.141580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.141644 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.141836 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.141836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.141837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.141982 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.142061 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.142173 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.142267 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:04:53.142879 24828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:53.143292 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.143181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46374)
0808 07:04:53.143231 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.143259 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:04:53.143623 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.143568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46384)
0808 07:04:53.143591 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.143611 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 07:04:53.143731 24828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:46257: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
W20260808 07:04:53.166572 24942 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:53.166790 24942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:53.166810 24942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:53.166829 24942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:53.168056 24942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:53.168097 24942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:53.168111 24942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:53.168123 24942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:53.168135 24942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:53.169605 24942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:41163,127.24.181.125:45729,127.24.181.124:46257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:46257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:53.169811 24942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:53.169979 24942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:53.174584 24948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:53.174602 24950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:53.174561 24952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:53.174868 24828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } attempt: 1
I20260808 07:04:53.174971 24942 server_base.cc:1034] running on GCE node
I20260808 07:04:53.175237 24942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:53.175326 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.175168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46386)
0808 07:04:53.175237 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.175278 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:04:53.175580 24942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:53.175704 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.175593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46398)
0808 07:04:53.175650 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.175680 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:04:53.175851 24828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:46257: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
I20260808 07:04:53.176754 24942 hybrid_clock.cc:648] HybridClock initialized: now 1786172693176732 us; error 35 us; skew 500 ppm
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:53.179296 24942 init.cc:377] Logged in from keytab as kudu/127.24.181.124@KRBTEST.COM (short username kudu)
I20260808 07:04:53.179747 24942 webserver.cc:460] Webserver started at http://127.24.181.124:45175/ using document root <none> and password file <none>
I20260808 07:04:53.179979 24942 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:53.180025 24942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:53.180171 24942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:53.181270 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data/instance:
uuid: "54cd7131975c49f1a9d7aa3cfde542fd"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "bebb527e506380225b84b47b38ae9e5b"
server_key_iv: "7ff5317bf5e6bbe0e61562f9effb53f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.181646 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal/instance:
uuid: "54cd7131975c49f1a9d7aa3cfde542fd"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "bebb527e506380225b84b47b38ae9e5b"
server_key_iv: "7ff5317bf5e6bbe0e61562f9effb53f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.183053 24942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:04:53.183832 24960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:53.184016 24942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:04:53.184118 24942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "54cd7131975c49f1a9d7aa3cfde542fd"
format_stamp: "Formatted at 2026-08-08 07:04:53 on dist-test-slave-nfb5"
server_key: "bebb527e506380225b84b47b38ae9e5b"
server_key_iv: "7ff5317bf5e6bbe0e61562f9effb53f5"
server_key_version: "encryptionkey@0"
I20260808 07:04:53.184222 24942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:53.192369 24930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } attempt: 2
I20260808 07:04:53.192759 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.192632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46410)
0808 07:04:53.192691 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.192730 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:04:53.193104 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.193012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46426)
0808 07:04:53.193056 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.193079 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:04:53.193183 24930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:46257: Network error: Client connection negotiation failed: client connection to 127.24.181.124:46257: connect: Connection refused (error 111)
I20260808 07:04:53.205157 24942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:53.207141 24942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:53.207330 24942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:53.211598 24942 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.124:46257
I20260808 07:04:53.211652 25029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.124:46257 every 8 connection(s)
I20260808 07:04:53.212023 24942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
I20260808 07:04:53.212883 25030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:53.214393 25030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:53.216624 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 24942
I20260808 07:04:53.216743 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172407597382-25301-0/minicluster-data/master-2/wal/instance
I20260808 07:04:53.216939 25301 external_mini_cluster.cc:1428] Setting key 949178547a49aa0871ae9e511284b471
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:53.224255 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.215057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.24.181.126:41163)
0808 07:04:53.215117 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.215119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.215318 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:04:53.215788 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.215789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.215799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.215850 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.216667 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.217408 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.218452 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.218657 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.218754 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.221032 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.221033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.221034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.221041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.222493 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.223021 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.223022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.223022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.223085 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.223391 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.223392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.223392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.223535 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.223609 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.224058 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.224134 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:04:53.224449 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.215061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:41163 (local address 127.0.0.1:48870)
0808 07:04:53.215215 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.215251 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.215705 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.215901 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.215904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.216140 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.216543 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.216549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.217493 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.217494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.218349 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.218352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.218759 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.218760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.218890 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.219457 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.219464 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.220935 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.222576 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.222583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.222585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.222907 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.223158 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.223159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.223160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.223309 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.223778 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.223785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.223909 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.223987 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.224164 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260808 07:04:53.225070 25030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:53.226548 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.217337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.24.181.124:46257)
0808 07:04:53.217499 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.217504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.217585 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:04:53.217650 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.217652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.217683 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.217786 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.218487 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.219291 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.220020 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.220328 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.220420 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.223316 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.223324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.223324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.223330 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.225197 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.225764 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.225767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.225768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.225830 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.226024 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.226025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.226025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.226190 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.226329 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.226378 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.226450 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:53.232137 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.225451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.24.181.125:45729)
0808 07:04:53.225503 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.225506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.225527 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:04:53.225551 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.225552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.225562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.225614 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.225822 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.226541 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.227365 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.227680 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.227747 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.229342 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.229344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.229345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.229352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.231016 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.231424 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.231425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.231425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.231476 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.231675 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.231676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.231676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.231817 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.231881 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.232018 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.232078 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:53.232195 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.225377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45729 (local address 127.0.0.1:56058)
0808 07:04:53.225432 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.225443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.225486 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.225615 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.225616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.225721 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.225766 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.225767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.226622 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.226623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.227306 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.227308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.227752 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.227754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.227837 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.228175 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.228180 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.229261 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.231096 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.231097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.231097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.231327 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.231545 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.231546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.231547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.231612 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.231876 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.231877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.231920 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.232000 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.232093 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:04:53.232816 25030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:53.235520 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.227514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.24.181.126:41163)
0808 07:04:53.227553 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.227555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.227563 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:53.227581 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.227581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.227590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.227649 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.228216 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.228631 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.229414 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.229622 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.229686 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.232281 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.232282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.232282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.232289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.233938 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.234291 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.234291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.234292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.234338 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.234550 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.234550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.234551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.234672 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.235291 (+ 619us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.235333 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.235440 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:53.239873 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.233327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46436)
0808 07:04:53.233373 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.233381 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.233416 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.233563 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.233564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.233701 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.233736 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.233738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.234553 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.234554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.235324 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.235327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.235820 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.235821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.235879 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.236235 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.236240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.237285 (+ 1045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.238854 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.238855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.238855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.239098 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.239303 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.239303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.239304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.239371 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.239664 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.239665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.239698 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.239751 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.239814 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:04:53.239990 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.233389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.24.181.124:46257)
0808 07:04:53.233436 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.233438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.233463 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:04:53.233480 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.233481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.233495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.233535 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.233812 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.234468 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.235402 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.235637 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.235709 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.237363 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.237365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.237365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.237372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.238776 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.239174 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.239175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.239176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.239230 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.239437 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.239438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.239439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.239583 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.239658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.239799 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.239881 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:53.241833 25030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: Bootstrap starting.
I20260808 07:04:53.242710 25030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:53.243091 25030 log.cc:824] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:53.243878 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.236137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56072 (local address 127.24.181.125:45729)
0808 07:04:53.236178 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.236181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.236189 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:53.236222 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.236223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.236232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.236287 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.236785 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.237300 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.238318 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.238514 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.238564 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.241211 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.241212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.241212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.241220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.242542 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.243006 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.243008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.243008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.243066 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.243260 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.243261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.243261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.243403 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.243635 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.243682 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.243770 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:53.243963 25030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: No bootstrap required, opened a new log
I20260808 07:04:53.245302 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:53.245846 25030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.245988 25030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54cd7131975c49f1a9d7aa3cfde542fd, State: Initialized, Role: FOLLOWER
I20260808 07:04:53.246079 25030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.246387 25030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:53.246344 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.246469 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.246119 24791 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f307794712ff41dbb3636aee7641a649"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48874
W20260808 07:04:53.248232 25052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:53.248318 25052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:53.251006 24828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } attempt: 2
I20260808 07:04:53.248524 24942 master_runner.cc:186] Error getting master registration for 127.24.181.126:41163: OK, instance_id { permanent_uuid: "f307794712ff41dbb3636aee7641a649" instance_seqno: 1786172692907054 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:53.251533 24942 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172692, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:53.258147 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.251314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.24.181.124:46257)
0808 07:04:53.251351 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.251353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.251360 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:04:53.251382 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.251383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.251390 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.251451 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.251742 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.252166 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.253140 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.253464 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.253526 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.255558 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.255560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.255560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.255567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.256980 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.257343 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.257345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.257345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.257405 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.257607 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.257608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.257609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.257744 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.257825 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.257980 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.258055 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:53.258147 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.251251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46444)
0808 07:04:53.251295 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.251304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.251339 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.251462 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.251463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.251634 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.251675 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.251677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.252258 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.252259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.253049 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.253051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.253540 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.253541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.253602 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.253969 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.253974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.255465 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.257085 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.257086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.257087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.257270 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.257474 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.257475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.257475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.257534 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.257819 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.257820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.257865 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.257955 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.258055 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:04:53.260069 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: Bootstrap starting.
I20260808 07:04:53.260720 24828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:53.261032 24828 log.cc:824] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:53.261826 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: No bootstrap required, opened a new log
I20260808 07:04:53.262854 24828 raft_consensus.cc:348] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.263048 24828 raft_consensus.cc:729] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f307794712ff41dbb3636aee7641a649, State: Initialized, Role: FOLLOWER
I20260808 07:04:53.263145 24828 consensus_queue.cc:260] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.263656 24828 sys_catalog.cc:564] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:53.263594 25058 sys_catalog.cc:455] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.263736 25058 sys_catalog.cc:458] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.263805 24791 raft_consensus.cc:482] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:04:53.263989 24791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:53.264804 24791 raft_consensus.cc:504] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.265535 24994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f307794712ff41dbb3636aee7641a649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "54cd7131975c49f1a9d7aa3cfde542fd"
I20260808 07:04:53.265657 24994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:53.266568 24994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f307794712ff41dbb3636aee7641a649 in term 1.
I20260808 07:04:53.267168 24756 leader_election.cc:302] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54cd7131975c49f1a9d7aa3cfde542fd, f307794712ff41dbb3636aee7641a649; no voters:
I20260808 07:04:53.267334 25058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:53.267467 25058 raft_consensus.cc:686] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [term 1 LEADER]: Becoming Leader. State: Replica: f307794712ff41dbb3636aee7641a649, State: Running, Role: LEADER
I20260808 07:04:53.267540 25058 consensus_queue.cc:237] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.268075 24791 leader_election.cc:288] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [CANDIDATE]: Term 1 election: Requested vote from peers 80dc241d985f4b629d1e673e529c0011 (127.24.181.125:45729), 54cd7131975c49f1a9d7aa3cfde542fd (127.24.181.124:46257)
I20260808 07:04:53.267995 25070 sys_catalog.cc:455] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f307794712ff41dbb3636aee7641a649. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
W20260808 07:04:53.269109 25069 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:53.269285 25069 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:53.266786 24727 master_runner.cc:186] Error getting master registration for 127.24.181.125:45729: OK, instance_id { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" instance_seqno: 1786172693049875 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:53.269440 24727 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:04:53.269544 25070 sys_catalog.cc:458] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:53.269896 25078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:53.270334 25078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:53.283950 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.270337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.24.181.125:45729)
0808 07:04:53.270387 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.270388 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.272201 (+ 1813us) server_negotiation.cc:407] Connection header received
0808 07:04:53.272313 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.272315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.272327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.272363 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.275338 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.276189 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.277348 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.277796 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.277886 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.279861 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.279862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.279863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.279871 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.282118 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.282529 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.282530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.282531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.282587 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.282799 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.282800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.282801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.282973 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.283534 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.283748 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.283848 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:04:53.284019 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.270010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.24.181.124:46257)
0808 07:04:53.270055 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.270057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.271037 (+ 980us) server_negotiation.cc:407] Connection header received
0808 07:04:53.271163 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.271164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.271177 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.271213 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.273228 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.274020 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.277383 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.277781 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.277842 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.280506 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.280508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.280508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.280516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.282267 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.282859 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.282860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.282861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.282960 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.283302 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.283303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.283303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.283470 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.283561 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.283853 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.283926 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:04:53.284304 24930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } attempt: 3
Aug 08 07:04:53 dist-test-slave-nfb5 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:53.290729 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.270713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.24.181.126:41163)
0808 07:04:53.270751 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.270753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.275072 (+ 4319us) server_negotiation.cc:407] Connection header received
0808 07:04:53.275174 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.275175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.275184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.275225 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.283018 (+ 7793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.283748 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.285260 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.285563 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.285601 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.287106 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.287108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.287108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.287115 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.288849 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.289446 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.289448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.289448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.289507 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.289874 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.289875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.289876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.290019 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.290111 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.290539 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.290621 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:04:53.291615 24994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [term 1 FOLLOWER]: Refusing update from remote peer f307794712ff41dbb3636aee7641a649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:53.291908 25058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:53.293320 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f307794712ff41dbb3636aee7641a649. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.293408 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.293690 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.284545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:46257 (local address 127.0.0.1:46472)
0808 07:04:53.284628 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.284637 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:53.284671 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:53.285177 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:53.285178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:53.285325 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:53.285357 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.285358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.286203 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.286204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.287032 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:53.287035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.287544 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.287545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:53.287611 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.288971 (+ 1360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:53.288976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:53.290534 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:53.292606 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.292606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.292607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.292798 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.292974 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:53.292992 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:53.292992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:53.293053 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:53.293359 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:53.293360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:53.293418 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:04:53.293485 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:04:53.293586 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 07:04:53.294072 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.284847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.24.181.124:46257)
0808 07:04:53.285060 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.285062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.285070 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:53.285103 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.285104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.285113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.285164 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.285448 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.286129 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.287109 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:53.287417 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:53.287494 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:53.290621 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:53.290623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:53.290623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:53.290629 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:53.292518 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.292870 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.292871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.292871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.292921 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:53.293129 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:53.293130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:53.293130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:53.293247 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:53.293307 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:53.293894 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 07:04:53.293967 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260808 07:04:53.295163 25058 sys_catalog.cc:455] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.295260 25058 sys_catalog.cc:458] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:53.295521 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.295594 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.295672 24930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: Bootstrap starting.
I20260808 07:04:53.295800 25084 mvcc.cc:205] Tried to move back new op lower bound from 7316163351717285888 to 7316163351624478720. Current Snapshot: MvccSnapshot[applied={T|T < 7316163351717285888}]
I20260808 07:04:53.296435 24930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:53.296741 24930 log.cc:824] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:53.297950 24930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: No bootstrap required, opened a new log
I20260808 07:04:53.298569 25085 mvcc.cc:205] Tried to move back new op lower bound from 7316163351717285888 to 7316163351624478720. Current Snapshot: MvccSnapshot[applied={T|T < 7316163351717285888}]
I20260808 07:04:53.300817 24930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.300962 24930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80dc241d985f4b629d1e673e529c0011, State: Initialized, Role: FOLLOWER
I20260808 07:04:53.301105 24930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } }
I20260808 07:04:53.301482 24930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:53.301527 24895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f307794712ff41dbb3636aee7641a649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "80dc241d985f4b629d1e673e529c0011"
I20260808 07:04:53.301626 24895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:53.302606 24895 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f307794712ff41dbb3636aee7641a649 in term 1.
I20260808 07:04:53.302819 24894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [term 1 FOLLOWER]: Refusing update from remote peer f307794712ff41dbb3636aee7641a649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:53.303249 25070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:53.301451 25090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.303442 25090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.304875 25078 catalog_manager.cc:1347] Generated new cluster ID: 66e7f3aeb386461ea86b27290af9e83b
I20260808 07:04:53.304927 25078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:53.305833 25090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f307794712ff41dbb3636aee7641a649. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.305943 25090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.306079 25106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.306159 25106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:53.311872 25103 catalog_manager.cc:1259] Loaded cluster ID: 66e7f3aeb386461ea86b27290af9e83b
I20260808 07:04:53.311925 25103 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: loading cluster ID for follower catalog manager: success
W20260808 07:04:53.312139 25103 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 07:04:53.312184 25103 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:53.318156 25070 sys_catalog.cc:455] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f307794712ff41dbb3636aee7641a649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f307794712ff41dbb3636aee7641a649" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 41163 } } peers { permanent_uuid: "80dc241d985f4b629d1e673e529c0011" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45729 } } peers { permanent_uuid: "54cd7131975c49f1a9d7aa3cfde542fd" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 46257 } } }
I20260808 07:04:53.318259 25070 sys_catalog.cc:458] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:53.318658 25078 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:53.319339 25078 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:53.325246 25078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f307794712ff41dbb3636aee7641a649: Generated new TSK 0
I20260808 07:04:53.325424 25078 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:04:53.355978 25035 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.354191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.24.181.124:46257)
0808 07:04:53.354237 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.354239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.355291 (+ 1052us) server_negotiation.cc:407] Connection header received
0808 07:04:53.355420 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.355421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.355432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.355464 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.355907 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46476: BlockingRecv error: recv got EOF from 127.0.0.1:46476 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:04:53.356760 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.354613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.24.181.125:45729)
0808 07:04:53.354656 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.354658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.356229 (+ 1571us) server_negotiation.cc:407] Connection header received
0808 07:04:53.356327 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.356328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.356340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.356381 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.356699 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56080: BlockingRecv error: recv got EOF from 127.0.0.1:56080 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:04:53.357532 24832 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.354988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.24.181.126:41163)
0808 07:04:53.355032 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.355034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.356994 (+ 1960us) server_negotiation.cc:407] Connection header received
0808 07:04:53.357095 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.357096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.357106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.357135 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.357466 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48894: BlockingRecv error: recv got EOF from 127.0.0.1:48894 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:04:53.359979 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.359189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.24.181.125:45729)
0808 07:04:53.359224 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.359225 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.359459 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:04:53.359567 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.359568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.359586 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.359616 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.359938 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56088: BlockingRecv error: recv got EOF from 127.0.0.1:56088 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:04:53.362982 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.362191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.24.181.125:45729)
0808 07:04:53.362225 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.362227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.362480 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:04:53.362573 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.362574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.362584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.362614 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.362941 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56090: BlockingRecv error: recv got EOF from 127.0.0.1:56090 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:04:53.366888 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.366110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.24.181.125:45729)
0808 07:04:53.366152 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.366153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.366397 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:04:53.366498 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.366499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.366509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.366538 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.366848 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56106: BlockingRecv error: recv got EOF from 127.0.0.1:56106 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:04:53.372870 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.372043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.24.181.125:45729)
0808 07:04:53.372079 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.372081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.372318 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:04:53.372434 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.372435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.372446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.372476 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.372816 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56118: BlockingRecv error: recv got EOF from 127.0.0.1:56118 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:04:53.382877 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.382062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56132 (local address 127.24.181.125:45729)
0808 07:04:53.382100 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.382102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.382335 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:04:53.382449 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.382451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.382461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.382490 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.382820 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56132: BlockingRecv error: recv got EOF from 127.0.0.1:56132 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:04:53.401438 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.400335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.24.181.125:45729)
0808 07:04:53.400384 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.400387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.400729 (+ 342us) server_negotiation.cc:407] Connection header received
0808 07:04:53.400865 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.400867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.400880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.400918 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.401377 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56148: BlockingRecv error: recv got EOF from 127.0.0.1:56148 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:04:53.435755 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.434899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.24.181.125:45729)
0808 07:04:53.434980 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.434982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.435142 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:04:53.435227 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.435228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.435240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.435312 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.435678 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56154: BlockingRecv error: recv got EOF from 127.0.0.1:56154 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 07:04:53.501736 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.500988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.24.181.125:45729)
0808 07:04:53.501090 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.501092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.501192 (+ 100us) server_negotiation.cc:407] Connection header received
0808 07:04:53.501259 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.501260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.501271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.501337 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.501662 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56156: BlockingRecv error: recv got EOF from 127.0.0.1:56156 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 07:04:53.631763 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.631031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56162 (local address 127.24.181.125:45729)
0808 07:04:53.631112 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.631114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.631255 (+ 141us) server_negotiation.cc:407] Connection header received
0808 07:04:53.631291 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.631293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.631305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.631378 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.631706 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56162: BlockingRecv error: recv got EOF from 127.0.0.1:56162 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 07:04:53.889894 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:53.889202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.24.181.125:45729)
0808 07:04:53.889271 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:53.889273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:53.889302 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:04:53.889381 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:53.889382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:53.889392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:53.889445 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:53.889828 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56172: BlockingRecv error: recv got EOF from 127.0.0.1:56172 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:04:54.248798 25052 catalog_manager.cc:1259] Loaded cluster ID: 66e7f3aeb386461ea86b27290af9e83b
I20260808 07:04:54.248884 25052 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: loading cluster ID for follower catalog manager: success
I20260808 07:04:54.249701 25052 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: acquiring CA information for follower catalog manager: success
I20260808 07:04:54.249971 25052 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54cd7131975c49f1a9d7aa3cfde542fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:54.313318 25103 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: acquiring CA information for follower catalog manager: success
I20260808 07:04:54.313649 25103 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80dc241d985f4b629d1e673e529c0011: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:54.405629 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.403183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.24.181.125:45729)
0808 07:04:54.403311 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:54.403313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:54.403374 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:04:54.403425 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:54.403426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:54.403480 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:54.403555 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:54.403701 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.404198 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.404945 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.405138 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.405189 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:54.405339 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:54.405450 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:54.405510 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:54.405545 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 07:04:54.405561 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260808 07:04:54.410169 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.407588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.24.181.124:46257)
0808 07:04:54.407722 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:54.407724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:54.407734 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:54.407819 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:54.407820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:54.407877 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:54.407965 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:54.408089 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.408581 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.409342 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.409566 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.409646 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:54.409818 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:54.409925 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:54.410014 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:54.410055 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:04:54.410075 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260808 07:04:54.414794 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.411939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.24.181.126:41163)
0808 07:04:54.412079 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:54.412081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:54.412118 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:04:54.412229 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:54.412231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:54.412290 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:54.412375 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:54.412465 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.413246 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.413999 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.414211 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.414293 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:54.414480 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:54.414561 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:54.414635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:54.414682 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 07:04:54.414705 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 07:04:54.415723 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.412004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.24.181.124:46257)
0808 07:04:54.412047 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:54.412049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:54.412223 (+ 174us) server_negotiation.cc:407] Connection header received
0808 07:04:54.412337 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:54.412338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:54.412350 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:54.412401 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:54.413359 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.413801 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.414981 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.415256 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.415333 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:54.415493 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:54.415525 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:54.415575 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:54.415629 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:04:54.415651 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:54.416136 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.411938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.24.181.125:45729)
0808 07:04:54.411979 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:54.411981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:54.412220 (+ 239us) server_negotiation.cc:407] Connection header received
0808 07:04:54.412279 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:54.412280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:54.412292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:54.412336 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:54.412539 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.413257 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.414975 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:54.415251 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:54.415328 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:54.415540 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:54.415576 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:54.415630 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:54.416063 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 07:04:54.416084 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:54.419431 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24727
I20260808 07:04:54.437382 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24836
I20260808 07:04:54.448222 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 24942
2026-08-08T07:04:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3005 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:54 dist-test-slave-nfb5 krb5kdc[25228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:54 dist-test-slave-nfb5 krb5kdc[25228](info): set up 2 sockets
Aug 08 07:04:54 dist-test-slave-nfb5 krb5kdc[25228](info): commencing operation
krb5kdc: starting...
W20260808 07:04:55.667142 25301 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/test-user.keytab.
Aug 08 07:04:55 dist-test-slave-nfb5 krb5kdc[25228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
Entry for principal kudu/127.24.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab.
I20260808 07:04:55.790331 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34513
--webserver_interface=127.24.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:55.872829 25248 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:55.873039 25248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:55.873057 25248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:55.873078 25248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:55.874154 25248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:55.874192 25248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:55.874207 25248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:55.874219 25248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:55.874231 25248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:55.875739 25248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.126.keytab
--principal=kudu/127.24.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.126:34513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:55.875931 25248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:55.876094 25248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:55.880961 25256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:55.880954 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:55.880999 25248 server_base.cc:1034] running on GCE node
W20260808 07:04:55.880954 25257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:55.881458 25248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:55.881765 25248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:55.882965 25248 hybrid_clock.cc:648] HybridClock initialized: now 1786172695882951 us; error 46 us; skew 500 ppm
Aug 08 07:04:55 dist-test-slave-nfb5 krb5kdc[25228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:55.885708 25248 init.cc:377] Logged in from keytab as kudu/127.24.181.126@KRBTEST.COM (short username kudu)
I20260808 07:04:55.886134 25248 webserver.cc:460] Webserver started at http://127.24.181.126:37395/ using document root <none> and password file <none>
I20260808 07:04:55.886370 25248 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:55.886442 25248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:55.886598 25248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:55.887792 25248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data/instance:
uuid: "484d3b81b4314e10a815d4c94a001f3b"
format_stamp: "Formatted at 2026-08-08 07:04:55 on dist-test-slave-nfb5"
server_key: "bf00277b8e663770cf70acd5600d9ca2"
server_key_iv: "079aff3cb658c8843f717af36521b81e"
server_key_version: "encryptionkey@0"
I20260808 07:04:55.888172 25248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance:
uuid: "484d3b81b4314e10a815d4c94a001f3b"
format_stamp: "Formatted at 2026-08-08 07:04:55 on dist-test-slave-nfb5"
server_key: "bf00277b8e663770cf70acd5600d9ca2"
server_key_iv: "079aff3cb658c8843f717af36521b81e"
server_key_version: "encryptionkey@0"
I20260808 07:04:55.889608 25248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:04:55.890400 25270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:55.890573 25248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:55.890679 25248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal
uuid: "484d3b81b4314e10a815d4c94a001f3b"
format_stamp: "Formatted at 2026-08-08 07:04:55 on dist-test-slave-nfb5"
server_key: "bf00277b8e663770cf70acd5600d9ca2"
server_key_iv: "079aff3cb658c8843f717af36521b81e"
server_key_version: "encryptionkey@0"
I20260808 07:04:55.890763 25248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:55.906476 25248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:55.908990 25248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:55.909183 25248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:55.913311 25248 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.126:34513
I20260808 07:04:55.913350 25356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.126:34513 every 8 connection(s)
I20260808 07:04:55.913729 25248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:55.914615 25358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:55.915755 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25248
I20260808 07:04:55.915879 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-0/wal/instance
I20260808 07:04:55.916092 25301 external_mini_cluster.cc:1428] Setting key 952a0d51a44c1d5ae55a86ff4a27b688
I20260808 07:04:55.916400 25358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:04:55 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.181.125@KRBTEST.COM; defaulting to no policy
I20260808 07:04:55.925995 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.917089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34513 (local address 127.0.0.1:43858)
0808 07:04:55.917257 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.917299 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:55.917746 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:55.918086 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:55.918089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:55.918392 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:55.918538 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:55.918543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:55.919625 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:55.919626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:55.920278 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:55.920280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:55.920789 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:55.920790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:55.920854 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:55.921431 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:55.921439 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:55.922769 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:55.924596 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:55.924602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:55.924604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:55.924794 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:55.925035 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:55.925036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:55.925036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:55.925135 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:55.925534 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:55.925542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:55.925618 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 07:04:55.925711 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:04:55.925841 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260808 07:04:55.925998 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.917120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.24.181.126:34513)
0808 07:04:55.917882 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:55.917886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:55.917896 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:55.917955 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:55.917956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:55.917971 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:55.918069 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:55.918654 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:55.919530 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:55.920393 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:55.920702 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:55.920828 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:55.922880 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:55.922888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:55.922890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:55.922902 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:55.924502 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:55.924892 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:55.924895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:55.924895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:55.924967 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:55.925228 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:55.925229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:55.925230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:55.925437 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:55.925559 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:55.925765 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:04:55.925922 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":741,"thread_start_us":77,"threads_started":1}
I20260808 07:04:55.926796 25358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.125@KRBTEST.COM" created.
I20260808 07:04:55.927175 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.927059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50888)
0808 07:04:55.927105 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.927151 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:04:55.927335 25273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
I20260808 07:04:55.927805 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.927680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50902)
0808 07:04:55.927737 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.927772 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:04:55.927976 25358 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45769: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
Entry for principal kudu/127.24.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab.
I20260808 07:04:55.941299 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:45769
--webserver_interface=127.24.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:55.977202 25358 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } attempt: 1
I20260808 07:04:55.977682 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.977550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50914)
0808 07:04:55.977611 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.977651 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:04:55.978055 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.977958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50926)
0808 07:04:55.978003 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.978035 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:04:55.978143 25358 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45769: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
W20260808 07:04:56.023942 25371 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:56.024161 25371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:56.024179 25371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:56.024230 25371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:56.025420 25371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:56.025465 25371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:56.025480 25371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:56.025492 25371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:56.025504 25371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:56.027072 25371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.125.keytab
--principal=kudu/127.24.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.125:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:56.027341 25371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:56.027570 25371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:56.032510 25384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:56.032541 25371 server_base.cc:1034] running on GCE node
W20260808 07:04:56.032523 25379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:56.032524 25377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:56.032927 25371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:56.033221 25371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:56.034389 25371 hybrid_clock.cc:648] HybridClock initialized: now 1786172696034383 us; error 39 us; skew 500 ppm
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:56.036887 25371 init.cc:377] Logged in from keytab as kudu/127.24.181.125@KRBTEST.COM (short username kudu)
I20260808 07:04:56.037355 25371 webserver.cc:460] Webserver started at http://127.24.181.125:36661/ using document root <none> and password file <none>
I20260808 07:04:56.037592 25371 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:56.037660 25371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:56.037824 25371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:56.038879 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data/instance:
uuid: "ee39b32630d7427ab9b6d90bbc5a841d"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9689a4415cdbab340f19e067517dbb17"
server_key_iv: "460648c95863b9933c880a10cf954609"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.039323 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal/instance:
uuid: "ee39b32630d7427ab9b6d90bbc5a841d"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9689a4415cdbab340f19e067517dbb17"
server_key_iv: "460648c95863b9933c880a10cf954609"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.040666 25371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:56.041373 25392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:56.041556 25371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:56.041663 25371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal
uuid: "ee39b32630d7427ab9b6d90bbc5a841d"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9689a4415cdbab340f19e067517dbb17"
server_key_iv: "460648c95863b9933c880a10cf954609"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.041738 25371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:56.046278 25358 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } attempt: 2
I20260808 07:04:56.046664 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.046543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50940)
0808 07:04:56.046601 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.046637 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:04:56.047026 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.046947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50954)
0808 07:04:56.046976 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.047000 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 07:04:56.047116 25358 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.125:45769: Network error: Client connection negotiation failed: client connection to 127.24.181.125:45769: connect: Connection refused (error 111)
I20260808 07:04:56.051577 25371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:56.053346 25371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:56.053510 25371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:56.057731 25371 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.125:45769
I20260808 07:04:56.057782 25463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.125:45769 every 8 connection(s)
I20260808 07:04:56.058167 25371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/data/info.pb
I20260808 07:04:56.059132 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:56.060689 25465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.065920 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25371
I20260808 07:04:56.066046 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-1/wal/instance
I20260808 07:04:56.066260 25301 external_mini_cluster.cc:1428] Setting key bca38e6b76f1811e2533ca4d7b57913d
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:56.071125 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.061387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.24.181.126:34513)
0808 07:04:56.061446 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.061448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.061661 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:04:56.062121 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.062122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.062133 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.062192 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.062808 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.063379 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.064361 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.064561 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.064618 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.067218 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.067220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.067220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.067226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.069101 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.069867 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.069868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.069868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.069938 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.070277 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.070279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.070279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.070408 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.070472 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.070904 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.071058 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:04:56.071362 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.061352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34513 (local address 127.0.0.1:43862)
0808 07:04:56.061537 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.061578 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.062042 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.062271 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.062274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.062515 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.062731 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.062736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.063462 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.063463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.064273 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.064275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.064640 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.064640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.064763 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.065315 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.065323 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.067111 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.069195 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.069227 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.069230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.069610 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.070026 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.070027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.070028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.070185 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.070489 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.070498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.070697 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.070827 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.071050 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260808 07:04:56.071983 25465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.181.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.181.124@KRBTEST.COM" created.
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:56.083604 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.072291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50970)
0808 07:04:56.072357 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.072370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.072422 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.072676 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.072677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.072853 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.072925 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.072927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.073903 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.073905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.074763 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.074766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.075355 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.075356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.075435 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.075991 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.075996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.080024 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.082307 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.082309 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.082309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.082505 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.082794 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.082795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.082795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.082857 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.083206 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.083207 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.083252 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.083337 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.083479 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:04:56.083647 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.072388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.24.181.125:45769)
0808 07:04:56.072531 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.072535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.072551 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:04:56.072567 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.072568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.072578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.072637 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.073010 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.073802 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.074861 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.075259 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.075322 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.080127 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.080138 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.080139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.080146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.082191 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.082654 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.082658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.082659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.082719 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.082940 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.082941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.082941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.083111 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.083218 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.083374 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.083541 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 07:04:56.084605 25465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.085057 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.084927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51800)
0808 07:04:56.084981 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.085034 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:04:56.085191 25394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
I20260808 07:04:56.085649 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.085552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51812)
0808 07:04:56.085601 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.085626 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:04:56.085762 25465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:35173: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
Entry for principal kudu/127.24.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab.
I20260808 07:04:56.096095 25301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNKFpcq/build/release/bin/kudu
/tmp/dist-test-taskNKFpcq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:35173
--webserver_interface=127.24.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.181.84:40205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:56.135046 25465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } attempt: 1
I20260808 07:04:56.135584 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.135433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51816)
0808 07:04:56.135507 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.135552 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:04:56.136018 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.135905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51830)
0808 07:04:56.135959 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.135995 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:04:56.136113 25465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:35173: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
I20260808 07:04:56.138266 25358 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } attempt: 3
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:56.146778 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.138675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.24.181.125:45769)
0808 07:04:56.138716 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.138718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.138737 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:04:56.138786 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.138787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.138797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.138857 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.139270 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.139819 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.140859 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.141091 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.141158 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.143482 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.143483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.143484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.143492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.145493 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.145888 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.145890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.145890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.145960 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.146188 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.146189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.146190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.146321 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.146402 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.146585 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.146674 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:56.146786 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.138621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50978)
0808 07:04:56.138675 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.138686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.138729 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.138887 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.138888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.139108 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.139186 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.139188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.139915 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.139917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.140751 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.140755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.141175 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.141176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.141269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.141800 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.141805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.143372 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.145596 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.145597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.145597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.145806 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.146042 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.146043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.146044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.146116 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.146406 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.146407 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.146461 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.146555 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.146681 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:04:56.148296 25358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.148663 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.148551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51834)
0808 07:04:56.148604 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.148633 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:04:56.149026 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.148930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51840)
0808 07:04:56.148978 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.149003 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:04:56.149123 25358 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:35173: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
I20260808 07:04:56.180292 25358 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } attempt: 1
I20260808 07:04:56.180856 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.180670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51844)
0808 07:04:56.180787 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.180822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 07:04:56.181232 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.181122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51856)
0808 07:04:56.181168 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.181194 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:04:56.181331 25358 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:35173: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
W20260808 07:04:56.182247 25477 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:04:56.182489 25477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:56.182528 25477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:56.182574 25477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:56.183768 25477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:56.183840 25477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:56.183871 25477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:56.183903 25477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:56.183933 25477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:56.185438 25477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.181.84:40205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.181.126:34513,127.24.181.125:45769,127.24.181.124:35173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/krb5kdc/kudu_127.24.181.124.keytab
--principal=kudu/127.24.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.181.124:35173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:04:56.185720 25477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:56.185956 25477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:56.190796 25489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:56.190815 25485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:56.190871 25486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:56.191155 25477 server_base.cc:1034] running on GCE node
I20260808 07:04:56.191411 25477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:56.191725 25477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:56.192893 25477 hybrid_clock.cc:648] HybridClock initialized: now 1786172696192867 us; error 41 us; skew 500 ppm
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:56.195595 25477 init.cc:377] Logged in from keytab as kudu/127.24.181.124@KRBTEST.COM (short username kudu)
I20260808 07:04:56.196094 25477 webserver.cc:460] Webserver started at http://127.24.181.124:35815/ using document root <none> and password file <none>
I20260808 07:04:56.196338 25477 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:56.196412 25477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:56.196561 25477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:56.197644 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data/instance:
uuid: "36fa84f566124a0992249ea61b3c571c"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9515f46da1840971dc66b6cfc5933174"
server_key_iv: "4a5a4ef8f440a5e032ec9ffb0eb5a193"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.198024 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal/instance:
uuid: "36fa84f566124a0992249ea61b3c571c"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9515f46da1840971dc66b6cfc5933174"
server_key_iv: "4a5a4ef8f440a5e032ec9ffb0eb5a193"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.199455 25477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:56.200189 25501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:56.200361 25477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:04:56.200469 25477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data,/tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal
uuid: "36fa84f566124a0992249ea61b3c571c"
format_stamp: "Formatted at 2026-08-08 07:04:56 on dist-test-slave-nfb5"
server_key: "9515f46da1840971dc66b6cfc5933174"
server_key_iv: "4a5a4ef8f440a5e032ec9ffb0eb5a193"
server_key_version: "encryptionkey@0"
I20260808 07:04:56.200544 25477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:56.204249 25465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } attempt: 2
I20260808 07:04:56.204623 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.204504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51862)
0808 07:04:56.204559 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.204595 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:04:56.204972 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.204878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51876)
0808 07:04:56.204921 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.204946 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:04:56.205058 25465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.181.124:35173: Network error: Client connection negotiation failed: client connection to 127.24.181.124:35173: connect: Connection refused (error 111)
I20260808 07:04:56.220450 25477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:56.222560 25477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:56.222730 25477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:56.227116 25477 rpc_server.cc:307] RPC server started. Bound to: 127.24.181.124:35173
I20260808 07:04:56.227156 25568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.181.124:35173 every 8 connection(s)
I20260808 07:04:56.227541 25477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/data/info.pb
I20260808 07:04:56.228410 25569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:56.230024 25569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.231686 25301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNKFpcq/build/release/bin/kudu as pid 25477
I20260808 07:04:56.231837 25301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNKFpcq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172407597382-25301-0/minicluster-data/master-2/wal/instance
I20260808 07:04:56.232092 25301 external_mini_cluster.cc:1428] Setting key bf3fde478bae235bf64c9ce5efb91b5e
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:56.240085 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.230757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.24.181.126:34513)
0808 07:04:56.230828 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.230830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.231057 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:04:56.231534 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.231536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.231549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.231637 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.232415 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.233135 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.234331 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.234629 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.234697 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.236963 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.236964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.236964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.236972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.238447 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.238940 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.238941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.238942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.239002 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.239327 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.239328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.239328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.239482 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.239560 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.239860 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.239966 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:04:56.240187 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.230727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.126:34513 (local address 127.0.0.1:43868)
0808 07:04:56.230902 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.230968 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.231428 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.231684 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.231688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.232032 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.232286 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.232293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.233229 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.233230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.234235 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.234239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.234719 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.234720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.234856 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.235379 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.235389 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.236870 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.238529 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.238538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.238541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.238849 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.239098 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.239099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.239099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.239244 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.239556 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.239562 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.239700 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.239819 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.239995 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260808 07:04:56.240885 25569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.242089 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.232523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.24.181.124:35173)
0808 07:04:56.232858 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.232862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.232885 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:04:56.232905 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.232907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.232930 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.233001 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.233762 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.234602 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.235677 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.235966 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.236064 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.238791 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.238799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.238799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.238805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.240844 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.241257 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.241260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.241261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.241334 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.241548 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.241548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.241549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.241728 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.241851 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.241955 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.242014 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":82,"threads_started":1}
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.126@KRBTEST.COM
I20260808 07:04:56.248158 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.241128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.125:45769 (local address 127.0.0.1:50992)
0808 07:04:56.241177 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.241187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.241225 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.241426 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.241427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.241586 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.241621 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.241623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.242492 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.242493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.243194 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.243197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.243588 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.243589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.243647 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.244037 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.244042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.245237 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.247084 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.247085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.247086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.247264 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.247487 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.247488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.247488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.247545 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.247844 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.247845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.247891 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.247969 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.248064 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:04:56.248183 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.241243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.24.181.125:45769)
0808 07:04:56.241290 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.241292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.241299 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:04:56.241331 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.241331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.241339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.241411 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.241705 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.242403 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.243279 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.243504 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.243578 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.245340 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.245341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.245342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.245350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.246994 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.247353 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.247354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.247355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.247415 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.247617 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.247618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.247618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.247756 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.247833 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.247999 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.248060 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:04:56.248530 25569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:56.250700 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.243030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.24.181.126:34513)
0808 07:04:56.243076 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.243079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.243104 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:04:56.243124 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.243125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.243133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.243202 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.243668 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.244268 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.245067 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.245288 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.245357 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.247848 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.247850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.247850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.247858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.249621 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.250001 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.250001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.250002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.250049 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.250256 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.250257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.250257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.250359 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.250497 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.250573 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.250637 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.124@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:56.255815 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.248803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51884)
0808 07:04:56.248854 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.248864 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.248904 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.249274 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.249275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.249407 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.249433 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.249434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.250293 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.250294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.251062 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.251064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.251499 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.251500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.251549 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.251946 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.251951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.253065 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.254714 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.254715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.254715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.254884 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.255135 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.255136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.255137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.255193 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.255435 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.255436 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.255473 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.255546 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.255649 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:04:56.255818 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.248865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.24.181.124:35173)
0808 07:04:56.248900 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.248902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.248910 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:56.249197 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.249198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.249217 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.249262 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.249510 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.250236 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.251156 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.251409 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.251485 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.253140 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.253141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.253142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.253149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.254605 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.255011 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.255012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.255012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.255069 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.255262 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.255263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.255263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.255369 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.255445 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.255598 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.255695 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":20}
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.181.125@KRBTEST.COM
I20260808 07:04:56.256472 25358 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } attempt: 2
I20260808 07:04:56.257566 25569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: Bootstrap starting.
I20260808 07:04:56.258450 25569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:56.258809 25569 log.cc:824] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:56.259265 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.251753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.24.181.125:45769)
0808 07:04:56.251794 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.251796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.251852 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:04:56.251919 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.251920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.251930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.251991 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.252427 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.252930 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.253863 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.254079 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.254155 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.256571 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.256572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.256572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.256580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.257981 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.258333 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.258334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.258335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.258390 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.258607 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.258608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.258608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.258759 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.259018 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.259066 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.259150 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:04:56.259805 25569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: No bootstrap required, opened a new log
I20260808 07:04:56.260993 25301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:56.261978 25569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.262099 25569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fa84f566124a0992249ea61b3c571c, State: Initialized, Role: FOLLOWER
I20260808 07:04:56.262183 25569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.262522 25576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.262691 25576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.262542 25569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:56.261924 25318 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "484d3b81b4314e10a815d4c94a001f3b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43880
W20260808 07:04:56.264778 25596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:56.264839 25596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172695, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.126@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:56.264986 25477 master_runner.cc:186] Error getting master registration for 127.24.181.126:34513: OK, instance_id { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" instance_seqno: 1786172695912187 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:56.268224 25477 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:04:56.270175 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.259438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.24.181.124:35173)
0808 07:04:56.262599 (+ 3161us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.262601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.262611 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:56.262631 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.262632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.262641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.262734 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.263144 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.263861 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.264843 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.265074 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.265156 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.267164 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.267166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.267166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.267173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.268892 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.269328 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.269329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.269329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.269389 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.269598 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.269599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.269599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.269758 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.269823 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.270000 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.270080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3123}
I20260808 07:04:56.270182 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.259234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51892)
0808 07:04:56.259317 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.259328 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.259365 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.262746 (+ 3381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.262748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.262980 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.263036 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.263039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.263954 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.263955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.264710 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.264714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.265155 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.265156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.265215 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.265617 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.265622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.267074 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.268984 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.268985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.268986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.269238 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.269458 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.269459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.269459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.269530 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.269833 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.269834 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.269885 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.269965 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.270073 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 07:04:56.271862 25358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: Bootstrap starting.
I20260808 07:04:56.272634 25358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:56.272970 25358 log.cc:824] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:56.273860 25358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: No bootstrap required, opened a new log
I20260808 07:04:56.275360 25358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.275509 25358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 484d3b81b4314e10a815d4c94a001f3b, State: Initialized, Role: FOLLOWER
I20260808 07:04:56.275593 25358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.275877 25600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.275950 25358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:56.275974 25600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.276081 25318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:04:56.276167 25318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:56.276949 25318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.277202 25318 leader_election.cc:288] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [CANDIDATE]: Term 1 election: Requested vote from peers ee39b32630d7427ab9b6d90bbc5a841d (127.24.181.125:45769), 36fa84f566124a0992249ea61b3c571c (127.24.181.124:35173)
W20260808 07:04:56.278417 25617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:56.278537 25617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:56.278664 25535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "484d3b81b4314e10a815d4c94a001f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "36fa84f566124a0992249ea61b3c571c"
I20260808 07:04:56.278803 25535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:56.279789 25535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 484d3b81b4314e10a815d4c94a001f3b in term 1.
I20260808 07:04:56.280037 25273 leader_election.cc:302] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fa84f566124a0992249ea61b3c571c, 484d3b81b4314e10a815d4c94a001f3b; no voters:
I20260808 07:04:56.279017 25248 master_runner.cc:186] Error getting master registration for 127.24.181.125:45769: OK, instance_id { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" instance_seqno: 1786172696056535 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:56.281524 25248 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:04:56.281591 25600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:56.281749 25600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [term 1 LEADER]: Becoming Leader. State: Replica: 484d3b81b4314e10a815d4c94a001f3b, State: Running, Role: LEADER
I20260808 07:04:56.281832 25600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.282936 25627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484d3b81b4314e10a815d4c94a001f3b. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.283036 25627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:56.287289 25629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:56.287480 25629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:56.292011 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.280250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.24.181.125:45769)
0808 07:04:56.280322 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.280324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.280607 (+ 283us) server_negotiation.cc:407] Connection header received
0808 07:04:56.280728 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.280729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.280741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.280771 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.281634 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.283224 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.284864 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.285161 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.285207 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.287954 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.287955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.287967 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.287975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.289839 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.290457 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.290458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.290458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.290520 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.290876 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.290877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.290877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.291068 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.291158 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.291521 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.291915 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:04:56.295701 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.282652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.24.181.126:34513)
0808 07:04:56.282688 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.282690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.288333 (+ 5643us) server_negotiation.cc:407] Connection header received
0808 07:04:56.288428 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.288429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.288439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.288468 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.289126 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.289870 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.290876 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.291211 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.291265 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.292719 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.292721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.292721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.292729 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.294458 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.294849 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.294851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.294851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.294939 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.295143 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.295143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.295144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.295268 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.295378 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.295529 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.295609 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:04:56.296197 25465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } attempt: 3
I20260808 07:04:56.300454 25535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [term 1 FOLLOWER]: Refusing update from remote peer 484d3b81b4314e10a815d4c94a001f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:56.300843 25627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 07:04:56 dist-test-slave-nfb5 krb5kdc[25228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172696, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.181.125@KRBTEST.COM for kudu/127.24.181.124@KRBTEST.COM
I20260808 07:04:56.301301 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.281949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.24.181.124:35173)
0808 07:04:56.282022 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.282023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.287113 (+ 5090us) server_negotiation.cc:407] Connection header received
0808 07:04:56.287178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.287179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.287192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.287260 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.293512 (+ 6252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.294524 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.295536 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.295834 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.295888 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.297388 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.297390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.297390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.297398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.299343 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.299720 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.299721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.299721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.299783 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.300003 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.300005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.300005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.300133 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.300223 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.301113 (+ 890us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.301190 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:04:56.302337 25576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484d3b81b4314e10a815d4c94a001f3b. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.302429 25576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.304459 25627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.304557 25627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:56.304654 25576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.304764 25576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.304760 25631 mvcc.cc:205] Tried to move back new op lower bound from 7316163364034568192 to 7316163363971969024. Current Snapshot: MvccSnapshot[applied={T|T < 7316163364034568192}]
I20260808 07:04:56.305215 25633 mvcc.cc:205] Tried to move back new op lower bound from 7316163364034568192 to 7316163363971969024. Current Snapshot: MvccSnapshot[applied={T|T < 7316163364034568192}]
I20260808 07:04:56.306092 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.296443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.181.124:35173 (local address 127.0.0.1:51912)
0808 07:04:56.296535 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.296545 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:56.296581 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:56.296860 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:56.296862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:56.297033 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:56.297080 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.297083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.298064 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.298065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.298905 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:56.298908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.299457 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.299458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:56.299514 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.299945 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:56.299951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:56.302724 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:56.304945 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.304946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.304946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.305158 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.305382 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:56.305383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:56.305384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:56.305440 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:56.305722 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:56.305723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:56.305774 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:04:56.305869 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:04:56.305990 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 07:04:56.306119 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.296556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51912 (local address 127.24.181.124:35173)
0808 07:04:56.296725 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.296727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.296734 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:04:56.296753 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.296754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.296763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.296828 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.297172 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.297959 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.299038 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:56.299369 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:56.299662 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:56.302828 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:56.302830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:56.302830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:56.302836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:56.304839 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.305253 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.305254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.305254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.305311 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:56.305508 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:56.305508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:56.305509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:56.305625 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:56.305706 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:56.305911 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:04:56.305997 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260808 07:04:56.307958 25465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: Bootstrap starting.
I20260808 07:04:56.308772 25465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:56.309085 25465 log.cc:824] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:56.309381 25629 catalog_manager.cc:1347] Generated new cluster ID: ab079e56fbb542258480b7ed102fc42e
I20260808 07:04:56.309439 25629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:56.310256 25465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: No bootstrap required, opened a new log
I20260808 07:04:56.313839 25465 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.314182 25465 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee39b32630d7427ab9b6d90bbc5a841d, State: Initialized, Role: FOLLOWER
I20260808 07:04:56.314397 25465 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } }
I20260808 07:04:56.315141 25426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:56.316057 25426 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [term 1 FOLLOWER]: Refusing update from remote peer 484d3b81b4314e10a815d4c94a001f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:56.316398 25600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:56.315168 25427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "484d3b81b4314e10a815d4c94a001f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ee39b32630d7427ab9b6d90bbc5a841d"
I20260808 07:04:56.317463 25427 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 484d3b81b4314e10a815d4c94a001f3b in term 1.
I20260808 07:04:56.315048 25465 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:56.320267 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.321013 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.321127 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484d3b81b4314e10a815d4c94a001f3b. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.321230 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.321303 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.321395 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:56.324385 25646 mvcc.cc:205] Tried to move back new op lower bound from 7316163364034568192 to 7316163363971969024. Current Snapshot: MvccSnapshot[applied={T|T < 7316163364034568192}]
W20260808 07:04:56.325186 25663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:56.325281 25663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:56.334964 25600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "484d3b81b4314e10a815d4c94a001f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484d3b81b4314e10a815d4c94a001f3b" member_type: VOTER last_known_addr { host: "127.24.181.126" port: 34513 } } peers { permanent_uuid: "ee39b32630d7427ab9b6d90bbc5a841d" member_type: VOTER last_known_addr { host: "127.24.181.125" port: 45769 } } peers { permanent_uuid: "36fa84f566124a0992249ea61b3c571c" member_type: VOTER last_known_addr { host: "127.24.181.124" port: 35173 } } }
I20260808 07:04:56.335090 25600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:56.343010 25629 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:56.343806 25629 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:56.359022 25629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 484d3b81b4314e10a815d4c94a001f3b: Generated new TSK 0
I20260808 07:04:56.359248 25629 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:04:56.390196 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.388386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.24.181.126:34513)
0808 07:04:56.388436 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.388439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.389454 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 07:04:56.389574 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.389575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.389587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.389619 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.390116 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43894: BlockingRecv error: recv got EOF from 127.0.0.1:43894 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:04:56.391034 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.388847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.24.181.125:45769)
0808 07:04:56.388889 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.388891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.390465 (+ 1574us) server_negotiation.cc:407] Connection header received
0808 07:04:56.390566 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.390567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.390578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.390608 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.390969 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51016: BlockingRecv error: recv got EOF from 127.0.0.1:51016 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:04:56.391801 25632 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.389160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.24.181.124:35173)
0808 07:04:56.389214 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.389216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.391250 (+ 2034us) server_negotiation.cc:407] Connection header received
0808 07:04:56.391380 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.391381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.391391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.391421 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.391739 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51914: BlockingRecv error: recv got EOF from 127.0.0.1:51914 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:04:56.394340 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.393518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.24.181.125:45769)
0808 07:04:56.393553 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.393555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.393762 (+ 207us) server_negotiation.cc:407] Connection header received
0808 07:04:56.393870 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.393871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.393890 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.393938 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.394294 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51024: BlockingRecv error: recv got EOF from 127.0.0.1:51024 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:04:56.397470 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.396677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.24.181.125:45769)
0808 07:04:56.396711 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.396713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.396961 (+ 248us) server_negotiation.cc:407] Connection header received
0808 07:04:56.397055 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.397056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.397066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.397094 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.397426 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51036: BlockingRecv error: recv got EOF from 127.0.0.1:51036 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:04:56.401405 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.400608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.24.181.125:45769)
0808 07:04:56.400661 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.400663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.400900 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:04:56.400995 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.400997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.401007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.401036 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.401360 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51048: BlockingRecv error: recv got EOF from 127.0.0.1:51048 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:04:56.407451 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.406569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51050 (local address 127.24.181.125:45769)
0808 07:04:56.406604 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.406606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.406847 (+ 241us) server_negotiation.cc:407] Connection header received
0808 07:04:56.407014 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.407015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.407026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.407057 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.407402 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51050: BlockingRecv error: recv got EOF from 127.0.0.1:51050 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:04:56.417714 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.416783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.24.181.125:45769)
0808 07:04:56.416823 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.416826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.417099 (+ 273us) server_negotiation.cc:407] Connection header received
0808 07:04:56.417229 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.417231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.417243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.417276 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.417659 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51064: BlockingRecv error: recv got EOF from 127.0.0.1:51064 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:04:56.436324 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.435279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.24.181.125:45769)
0808 07:04:56.435333 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.435335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.435678 (+ 343us) server_negotiation.cc:407] Connection header received
0808 07:04:56.435784 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.435786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.435798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.435829 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.436261 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51080: BlockingRecv error: recv got EOF from 127.0.0.1:51080 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:04:56.470530 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.469682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.24.181.125:45769)
0808 07:04:56.469734 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.469737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.469901 (+ 164us) server_negotiation.cc:407] Connection header received
0808 07:04:56.469975 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.469976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.469989 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.470065 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.470451 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51088: BlockingRecv error: recv got EOF from 127.0.0.1:51088 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:04:56.536595 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.535772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.24.181.125:45769)
0808 07:04:56.535854 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.535856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.535957 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:04:56.536064 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.536065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.536077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.536143 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.536529 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51096: BlockingRecv error: recv got EOF from 127.0.0.1:51096 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 07:04:56.666488 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.665807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.24.181.125:45769)
0808 07:04:56.665901 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.665903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.665981 (+ 78us) server_negotiation.cc:407] Connection header received
0808 07:04:56.666014 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.666015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.666026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.666094 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.666420 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51110: BlockingRecv error: recv got EOF from 127.0.0.1:51110 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 07:04:56.924414 25473 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:56.923791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.24.181.125:45769)
0808 07:04:56.923870 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:56.923872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:56.923884 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:56.923945 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:56.923946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:56.923956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:56.924019 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:56.924349 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51116: BlockingRecv error: recv got EOF from 127.0.0.1:51116 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:04:57.265324 25596 catalog_manager.cc:1259] Loaded cluster ID: ab079e56fbb542258480b7ed102fc42e
I20260808 07:04:57.265429 25596 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: loading cluster ID for follower catalog manager: success
I20260808 07:04:57.266240 25596 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: acquiring CA information for follower catalog manager: success
I20260808 07:04:57.266557 25596 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36fa84f566124a0992249ea61b3c571c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:57.325879 25663 catalog_manager.cc:1259] Loaded cluster ID: ab079e56fbb542258480b7ed102fc42e
I20260808 07:04:57.325966 25663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: loading cluster ID for follower catalog manager: success
I20260808 07:04:57.326814 25663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: acquiring CA information for follower catalog manager: success
I20260808 07:04:57.327175 25663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee39b32630d7427ab9b6d90bbc5a841d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:57.440378 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.437684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.24.181.125:45769)
0808 07:04:57.437807 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:57.437810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:57.437840 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:04:57.437907 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:57.437908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:57.437974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:57.438066 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:57.438213 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.438718 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.439540 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.439747 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.439833 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:57.440014 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:57.440121 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:57.440211 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:57.440251 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:04:57.440273 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260808 07:04:57.444375 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.442152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.24.181.124:35173)
0808 07:04:57.442256 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:57.442258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:57.442265 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:04:57.442298 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:57.442298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:57.442332 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:57.442399 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:57.442506 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.443038 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.443713 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.443915 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.443962 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:57.444080 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:57.444168 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:57.444262 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:57.444295 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:04:57.444313 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":56,"threads_started":1}
I20260808 07:04:57.448805 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.446074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.24.181.125:45769)
0808 07:04:57.446119 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:57.446121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:57.446216 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:04:57.446335 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:57.446336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:57.446346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:57.446402 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:57.446621 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.447239 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.448134 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.448311 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.448373 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:57.448549 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:57.448580 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:57.448629 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:57.448713 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 07:04:57.448733 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:04:57.449450 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.446120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.24.181.126:34513)
0808 07:04:57.446421 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:57.446423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:57.446432 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:57.446477 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:57.446478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:57.446535 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:57.446640 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:57.446741 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.447551 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.448471 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.448802 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.448878 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:57.449107 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:57.449211 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:57.449282 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:57.449340 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:04:57.449361 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":62,"threads_started":1}
I20260808 07:04:57.450366 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.446421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.24.181.124:35173)
0808 07:04:57.446479 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:57.446481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:57.446526 (+ 45us) server_negotiation.cc:407] Connection header received
0808 07:04:57.446600 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:57.446601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:57.446610 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:57.446659 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:57.446814 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.447550 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.449606 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:57.449896 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:57.449976 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:57.450114 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:57.450146 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:57.450199 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:57.450270 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:04:57.450287 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:04:57.451373 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25248
I20260808 07:04:57.466063 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25371
I20260808 07:04:57.473699 25301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNKFpcq/build/release/bin/kudu with pid 25477
2026-08-08T07:04:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3026 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6031 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289886 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST